Commit graph

672 commits

Author SHA1 Message Date
yuneng-jiang
bff3590dd0 Update sidebar for customer usage 2025-12-06 14:21:18 -08:00
Cesar Garcia
8ccfaa21de
docs: add Microsoft GraphRAG to projects using LiteLLM (#17616)
* docs: add Microsoft GraphRAG to projects using LiteLLM

* docs: add arXiv paper link for GraphRAG

* docs: add GraphRAG to sidebar

* Update projects in sidebars.js

Reordered items in the projects list to include 'GraphRAG'.
2025-12-06 13:47:46 -08:00
Ishaan Jaff
a9b654224e
1.80.8 RC docs (#17605)
* stash docs

* docs fix

* doc fix

* docs fix
2025-12-06 10:40:00 -08:00
Krrish Dholakia
497856e1e3 docs: document multi tenant architecture 2025-12-06 09:27:30 -08:00
Sameer Kankute
558c8f92d1
Merge pull request #17519 from BerriAI/litellm_cursor_integration
Add support for cursor BYOK with its own configuration
2025-12-05 22:23:45 +05:30
Sameer Kankute
c8fbcc7f1c add tutorial as well 2025-12-05 12:32:23 +05:30
Sameer Kankute
acc0b5fe27
Merge pull request #17362 from BerriAI/litellm_vertex-bge-cherrypick
[Feat] VertexAI - Add BGE Embeddings support
2025-12-05 11:53:42 +05:30
Sameer Kankute
a6006e698c Add support for cursor BYOK with its own configuration 2025-12-05 09:34:49 +05:30
Ishaan Jaffer
4f3b843efe docs openai 2025-12-04 18:32:23 -08:00
Ishaan Jaff
fadfbb13d3
[Docs] A2a - Permission management (#17515)
* docs add a2a gateway + mcp gateway

* docs a2a permissions

* docs a2a permission

* docs

* docs a2a

* docs a2a

* add new img

* docs agent permissions
2025-12-04 17:29:47 -08:00
Krrish Dholakia
5aeba81538 docs(multi_tenant_architecture.md): add new architecture doc 2025-12-04 11:13:50 -08:00
Ishaan Jaff
e4f954b354
[Docs] Agent Gateway (#17454)
* init litellm A2a client

* simpler a2a client interface

* test a2a

* move a2a invoking tests

* test fix

* ensure a2a send message is tracked n logs

* rename tags

* add streaming handlng

* add a2a invocation

* add a2a invocation i cost calc

* test_a2a_logging_payload

* update invoke_agent_a2a

* test_invoke_agent_a2a_adds_litellm_data

* add A2a agent

* fix endpoints on A2a

* UI allow testing a2a endpoints

* add agent imgs

* add a2a as an endpoint

* add a2a

* docs a2a invoke

* docs a2a

* docs A2a invoke
2025-12-03 18:57:41 -08:00
Cesar Garcia
5e791464af
docs: add Microsoft Agent Lightning to projects (#17422)
Add Agent Lightning, Microsoft's open-source framework for training
AI agents with RL, APO, and SFT. Uses LiteLLM Proxy for LLM routing
and trace collection.
2025-12-03 09:07:02 -08:00
Sameer Kankute
52090c3f3e
Merge pull request #17350 from BerriAI/litellm_rag_chat_completion_api
Add ragflow support for chat completions API
2025-12-03 13:29:32 +05:30
Cesar Garcia
86350fe6d7
docs: add Google ADK and Harbor to projects (#17352)
Both frameworks integrate with LiteLLM:
- Google ADK uses LiteLLM for model-agnostic agent building
- Harbor uses LiteLLM for agent evaluation across providers
2025-12-02 22:27:04 -08:00
Sameer Kankute
a0819d6df0
Merge branch 'main' into litellm_vertex-bge-cherrypick 2025-12-03 08:37:04 +05:30
Ishaan Jaff
6c188c5ae2
[Feat] New model/provider - Adds support for Google Cloud Chirp3 HD on /speech (#17391)
* docs vertex tts

* place vertex ai types in file

* use VertexAITextToSpeechConfig

* use vertex_voice_dict

* refactor docs

* docs vertex ai chirp

* TestVertexAITextToSpeechConfig

* new provider vertex ai chirp3

* test_litellm_speech_vertex_ai_chirp

* add vertex_ai/chirp cost trackign
2025-12-02 15:36:23 -08:00
Ishaan Jaff
db6c6eea89
[Docs] Add guide on how to debug gateway error vs provider error (#17387)
* add error diagnosis

* docs error diagnosis
2025-12-02 14:10:00 -08:00
Ishaan Jaffer
88b2cfc789 docs cleanup 2025-12-02 21:59:02 +05:30
Sameer Kankute
831ad45c4d Add ragflow support 2025-12-02 18:18:08 +05:30
Cesar Garcia
965406c643
feat(provider): add Z.AI (Zhipu AI) as built-in provider (#17307)
* feat(provider): add Z.AI (Zhipu AI) as built-in provider

Add support for Z.AI GLM models as a native OpenAI-compatible provider.

- Add "zai" to openai_compatible_providers list
- Add ZAI enum to LlmProviders
- Add provider URL resolution for https://api.z.ai/api/paas/v4
- Add 8 GLM models with pricing to model cost maps:
  - glm-4.6 (200K context, $0.6/$2.2 per 1M tokens)
  - glm-4.5, glm-4.5v, glm-4.5-x, glm-4.5-air, glm-4.5-airx
  - glm-4-32b-0414-128k
  - glm-4.5-flash (free tier)
- Add unit tests for provider integration

Closes #17289

* docs: add Z.AI provider documentation

- Add zai.md with usage examples, model list, and pricing
- Add to sidebars.js navigation
2025-12-01 19:56:47 -08:00
Krish Dholakia
b6d6f834e0
(feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo (#17175)
* feat(generic_guardrail_api.py): new generic api for guardrails

Allows guardrail providers to work with litellm for guardrails without needing to make a PR to LiteLLM

* docs(generic_guardrail_api.md): document new generic guardrail api

* Fix: Improve PII detection and guardrail API integration

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* feat: correctly extract raw request from guardrail api

* docs(generic_guardrail_api.md): document this is a beta feature

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-12-01 14:29:52 -08:00
Ishaan Jaff
edfc35ddac
[Feature]: Add Provider publicai.co (#17230)
* init PublicAIChatConfig

* add publicai

* init public ai

* add publicai

* add publicai/swiss-ai models etc
2025-11-27 13:08:03 -08:00
Ishaan Jaff
d612d71ef4
[Feat] Add guardrails for pass through endpoints (#17221)
* add PassThroughGuardrailsConfig

* init JsonPathExtractor

* feat PassthroughGuardrailHandler

* feat pt guardrails

* pt guardrails

* add Pass-Through Endpoint Guardrail Translation

* add PassThroughEndpointHandler

* execute simple guardrail config and dict settings

* TestPassthroughGuardrailHandlerNormalizeConfig

* add passthrough_guardrails_config on litellm logging obj

* add LiteLLMLoggingObj to base trasaltino

* cleaner _get_guardrail_settings

* update guardrails settings

* docs pt guardrail

* docs Guardrails on Pass-Through Endpoints

* fix typing

* fix typing

* test_no_fields_set_sends_full_body

* fix typing

* Potential fix for code scanning alert no. 3834: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-11-27 12:06:53 -08:00
Cesar Garcia
5197380dd2
docs: add OpenAI Agents SDK to projects (#17203)
* docs: add OpenAI Agents SDK to projects

Add documentation for OpenAI Agents SDK which has an official
LiteLLM extension enabling 100+ LLM providers in multi-agent workflows.

* Update project items in sidebars.js
2025-11-27 09:47:26 -08:00
Krish Dholakia
1cb5fcddba
make generic api OSS + support multiple generic API's (#17152)
* feat(generic_api_callback.py): make generic api OSS + support multiple generic API's

Enables https://github.com/BerriAI/litellm/pull/17094#discussion_r2562832967

* feat(callback_utils.py): support custom generic api callbacks

* feat(generic_api_callback.py): support specifying which event types to run the generic api for

* fix(litellm_logging.py): log system prompt for anthropic messages

* feat(generic_api_callback.py): support generic api compatible api's - e.g. rubrik agent cloud

* docs(sidebars.js): document new OSS generic api

* docs(generic_api.md): document new OSS Generic API

* docs(custom_webhook_api.md): document custom webhook api integration tutorial

* docs(custom_webhook_api.md): cleanup

* docs(custom_webhook_api.md): document what get's logged to custom webhook api

* Refactor: Pass callback config to GenericAPILogger

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* Fix: Handle empty messages list in logging payload

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* Checkpoint before follow-up message

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* feat: Cache GenericAPILogger instances to improve performance

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-11-26 18:38:38 -08:00
Ishaan Jaff
d987593855
[Feat] Add audio transcriptions for WatsonX (#17160)
* feat: add IBMWatsonXAudioTranscriptionConfig

* init IBMWatsonXAudioTranscriptionConfig

* add inits

* test_watsonx_transcription_url_and_headers

* docs watsox audio

* docs watsonx audi

* docs
2025-11-26 14:15:49 -08:00
Ishaan Jaff
5c192a23c3
[Feat] Add new RAG API on LiteLLM AI Gateway (#17109)
* init RAG api types

* add RAG endpoints

* init main.py for RAG ingest API

* init RecursiveCharacterTextSplitter

* add BaseRAGIngestion

* fix OpenAIRAGIngestion

* fix img handler

* init OpenAIRAGIngestion

* init BedrockRAGIngestion

* init BedrockRAGIngestion

* init rag tests

* init BedrockVectorStoreOptions

* implement BedrockRAGIngestion

* add BaseRAGAPI

* add endpoint for RAG ingest

* add ingest RAG endpoints

* add test doc

* add parse_rag_ingest_request

* update endpoints

* docs add docs for new RAG API

* fix qa check

* fix linting

* docs ficx

* docs

* add max depth checks

* docs anthropic
2025-11-25 17:54:29 -08:00
Ishaan Jaff
be712908a3
[Feat] Add OpenAI compatible bedrock imported models. - qwen etc (#17097)
* test_bedrock_openai_imported_model

* AmazonBedrockOpenAIConfig

* add openai route for bedrock

* docs fix

* fix code qa check
2025-11-25 12:20:39 -08:00
Ishaan Jaff
d2b90e6644
[Docs] Claude Skills API (#17052)
* add skillls api

* init doc

* docs fix

* skills API
2025-11-24 15:32:03 -08:00
Krish Dholakia
c368b4c180
Document model pricing contribution process (#17031)
* Add doc for adding model pricing and context window

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* Refactor model pricing documentation to include sample spec and examples

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-11-24 08:28:24 -08:00
Ishaan Jaff
f3cd0b0bc4
docs - ai gateway prompt management (#16990)
* docs prompt management

* docs

* add prompt mgmt imgs

* docs fix

* docs prompt management

* docs fix

* docs prompt versions

* docs fix
2025-11-22 18:49:29 -08:00
Krish Dholakia
c966c122ad
feat: Add Presidio PII masking tutorial (#16969)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-11-22 15:45:43 -08:00
Cesar Garcia
6810e0699b
docs: Add mini-swe-agent to Projects built on LiteLLM (#16971)
* docs: Add mini-swe-agent to projects page

Add mini-swe-agent to the documentation projects page.
mini-swe-agent is a minimal AI coding agent that resolves >70% of
GitHub issues in SWE-bench, built on LiteLLM for model flexibility.

- Added projects/mini-swe-agent.md documentation
- Updated sidebars.js to include mini-swe-agent in projects list

* docs: Update Singularity to Apptainer in mini-swe-agent.md
2025-11-22 10:45:29 -08:00
yuneng-jiang
1ebe1fea37
Docs for Model Compare UI and Org Usage (#16928)
* Docs for Model Compare UI and Org Usage

* Fix typo in img path and add Model Compare to sidebars.js

* Updated to remove from 1.80 writeup
2025-11-21 16:45:55 -08:00
Ishaan Jaff
8e318dd06c
[Feat] New LLM Provider - Docker Model Runner (#16948)
* add DOCKER_MODEL_RUNNER

* add DockerModelRunnerChatConfig Transorm

* add docker_model_runner

* add docker_model_runner

* docs docker model runner

* add DockerModelRunnerChatConfig

* add docker_model_runner to providers

* test_completion_hits_correct_url_and_body

* fix sidebar

* TestDockerModelRunnerIntegration

* test_completion_with_custom_engine_and_host

* docs docker model runner

* docs fix
2025-11-21 16:09:32 -08:00
Krrish Dholakia
208027dc71 docs(ui.md): reorder ui page 2025-11-19 19:17:54 -08:00
Krish Dholakia
39917f2ff7
Agents - AI Hub, make agents discoverable on model hub page for internal discovery (#16678)
* feat: initial commit adding agent hub to ui

* feat: add viewable agent hub

* feat: working support for making both config + db agents public via new 'public_agent_groups' list

* fix: agents.py

fix types

* feat: working PATCH endpoint for UI changes

* feat: add new agents panel with working crud

* refactor: refactor to show created_at on be/fe

* style: align new page with the agents table

* style: more style alignment logic

* feat: return if agent is public or not in /v1/agents

* feat: initial commit adding ui flow for making agents discoverable

* feat: new batch make public endpoint

* feat(public_model_hub.tsx): show public agents on public model hub table page

* fix(public_model_hub.tsx): add code examples for using the agent in a2a

* fix: fix indicating if agent has already been made public

* docs: document expected spec for agents is A2A

* docs: add agent hub docs

* docs: document making agents discoverable

* docs: add demo video to docs

* fix: fix ui linting errors

* fix: update tests
2025-11-15 18:25:16 -08:00
Sameer Kankute
a2e3b942dc
Vector store files Stable Release (#16643)
* Add support for vector store files endpoints (#16490)

* Add base code for vector store integration

* fix azure related tests and linting error

* fix mypy errors

* Add vector store files documentation

* fix mapped tests

* Add bytedance and ideogram support in fal ai (#16636)

* Add fal ai flux pro v1.1 support (#16578)

* Add fal ai flux pro v1.1 support

* Add tests and docs

---------

Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
2025-11-15 13:00:33 -08:00
Ishaan Jaff
8a43fbe8f7
Revert "[Feat] VertexAI - Add BGE Embeddings support (#16033)" (#16677)
This reverts commit 7133488282.
2025-11-14 17:41:06 -08:00
Ishaan Jaff
bffc36794f
docs fix spend tracking (#16675) 2025-11-14 17:22:21 -08:00
Ishaan Jaff
efa4ec9294
[Docs] Add docs on APIs for model access management (#16673)
* docs access groups

* docs ui for access groups

* fix code snippets

* docs model access
2025-11-14 17:07:54 -08:00
Krrish Dholakia
266744a5bd docs: add contribution guide for new guardrails 2025-11-13 22:29:42 -08:00
Ishaan Jaff
7133488282
[Feat] VertexAI - Add BGE Embeddings support (#16033)
* Support for Custom Vertex AI Models via PSC Endpoint with api_base (#15953)

* Support for Custom Vertex AI Models via PSC Endpoint with api_base

* Add docs related psc

* remove not needed files

* remove print statemnt

* fix mypy errors

* add TextEmbeddingBGEInput

* add VertexBGEConfig

* add BGE handling

* test_vertex_ai_bge_embedding_with_custom_api_base

* fix request transform vertex BGE

* test_vertex_ai_bge_embedding_with_custom_api_base

* tes BGE

* test_is_bge_model_detection

* docs cleanup

* handling BGE URL

* fix VertexBGEConfig

* test_vertex_ai_bge_with_endpoint_id_pattern

* docs vertex BGE

* docs

* docs fix

* fix VertexAIModelRoute

* from ..common_utils import VertexAIError, get_vertex_base_model_name
add

* fix VertexAIGemmaModels

* fix get_vertex_base_model_name

* test_vertex_ai_bge_psc_endpoint_url_construction

---------

Co-authored-by: Sameer Kankute <sameer@berri.ai>
2025-11-13 12:41:00 -08:00
Ishaan Jaff
b30439257b
[Feat] Add RunwayML Img Gen API support (#16557)
* TestRunwaymlImageGeneration

* fix RUNWAYML

* rename

* fix rename

* get_runwayml_image_generation_config

* get_runwayml_image_generation_config

* TestRunwaymlImageGeneration

* add RUNWAYML_POLLING_TIMEOUT

* fix rnwayml transform img gen

* runwayml_image_cost_calculator

* runwayml_image_cost_calculator

* docs runwayml

* fix runwayML polling

* test_get_first_default_fallback
2025-11-12 18:20:14 -08:00
Krrish Dholakia
a05ad46394 feat: add litellm cloud self-serve to docs 2025-11-12 17:13:14 -08:00
Krrish Dholakia
c6d2714c52 docs(model_access_guide.md): document how model access works on litellm 2025-11-12 13:43:13 -08:00
Ishaan Jaff
50b5cf5215
[Feat] New Provider - Add RunwayML Provider for video generations (#16505)
* add RUNWAYML

* init folders

* add RunwayMLVideoConfig

* add RUNWAYML_DEFAULT_API_VERSION

* add RunwayMLVideoConfig

* fix getting status

* add async_transform_video_content_response

* add runwayml transform_video_content_response

* fix config.yaml

* add runwayml docs

* add runwayml to videos

* docs runwayml video gen

* add new models to model cost map

* TestRunwayMLVideoTransformation

* fix linting errors
2025-11-11 18:48:23 -08:00
jwang-gif
443bada425
Add Zscaler AI Guard hook (#15691)
* Add Zscaler AI Guard hook

Co-authored-by: Angela Tao <atao@zscaler.com>

* Fix lint error, update document

* Fix lint error, update document

* update document

* fix mypy type error

* fix mypy issue

* fix test

* fix test

* improve document

* remove unuseful code

* use litellm httphandler

* update test cases

* revover guardrail_initializers.py and guardrail_registry.py

* remove unuse import

* app apply_guardrail

* remove functions repleased by apply_guardrail, update test and doc

* remove functions repleased by apply_guardrail, update test and doc

---------

Co-authored-by: Angela Tao <atao@zscaler.com>
2025-11-11 15:34:27 -08:00
‮Artem
a97676b8ae
Add softgen to projects that are using litellm (#16423)
* Add Softgen project documentation

* Add Softgen project to sidebars

* Update Softgen
2025-11-10 11:09:29 -08:00