Commit graph

33248 commits

Author SHA1 Message Date
yuneng-jiang
b29c0bfb8f
Update litellm/proxy/spend_tracking/spend_management_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-12 21:52:04 -08:00
Sameer Kankute
bd27874786 Add migration script for the new table 2026-02-13 11:14:01 +05:30
yuneng-jiang
ae44022a40 Fixing spend tests 2026-02-12 21:37:19 -08:00
yuneng-jiang
21b38a4030 Merge pull request #20720 from OrionCodeDev/fix-spend-logs
fix-spend-logs
2026-02-12 21:37:00 -08:00
Sameer Kankute
d29981ecc4 Add support for managed file ids 2026-02-13 10:52:48 +05:30
Sameer Kankute
be69ba270e Add using managed vector store creds for vector store files endpoint 2026-02-13 10:33:49 +05:30
Harshit Jain
7f6563f1a6
fix: openai moderation guardrails (#20718)
* fix: openai moderation guardrails

* adds missing import

* mv: test file to right place
2026-02-12 21:03:11 -08:00
Otavio Brito
b1a67666ea
refactor: reuse get_instance_fn in initialize_custom_guardrail - allow module level import (#20917) 2026-02-12 20:53:47 -08:00
Rahul Dhanawade
a58f18872b
Fix #20562: Correct Bedrock Claude Opus 4.6 model IDs (#20564) 2026-02-12 20:48:25 -08:00
The Mavik
99b4d17ee8
fix: guard against None litellm_metadata in batch logging (#20832)
When litellm_metadata is explicitly set to None in litellm_params,
`dict.get("litellm_metadata", {})` returns None (not the default {}),
because the key exists. The subsequent .get() call on None raises
`AttributeError: 'NoneType' object has no attribute 'get'`.

Use `or {}` instead, consistent with line 4924 in the same file.

Fixes #15836
2026-02-12 20:45:30 -08:00
Lei Nie
68d2306dd4
feat(vertex_ai): preserve usageMetadata in _hidden_params (#20559)
* fix: allow Management keys to access user/daily/activity and team/daily/activity

* feat(vertex): surface trafficType via generic provider_specific_fields in Responses API

Extract Vertex AI's trafficType from usageMetadata in both streaming and
non-streaming paths, storing it in _hidden_params["provider_specific_fields"].

The Responses API transformation layer generically passes any
_hidden_params["provider_specific_fields"] dict to the ResponsesAPIResponse,
avoiding provider-specific logic in the bridge.

Also fix stream_chunk_builder to propagate _hidden_params from the last
streaming chunk to the rebuilt ModelResponse, ensuring provider metadata
survives the chunk→response rebuild.

---------

Co-authored-by: naaa760 <neh6a683@gmail.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
2026-02-12 20:44:59 -08:00
Cesar Garcia
20ae67a3ba
fix(vertex): map IMAGE_PROHIBITED_CONTENT finish reason to content_filter (#20524)
Closes #20357
2026-02-12 20:39:55 -08:00
Sameer Kankute
26c0624fd7 Add managed vector store hooks 2026-02-13 10:03:31 +05:30
Sameer Kankute
c04bef376b Add router for vector store and search vector 2026-02-13 10:02:38 +05:30
Cesar Garcia
69f78ae64f
cleanup(model_prices): remove 39 deprecated OpenRouter models (#20786)
These models return 404 ("No endpoints found") from OpenRouter's API
and are no longer listed in /api/v1/models.

Fixes #20521
2026-02-12 20:32:00 -08:00
Cesar Garcia
50ce7c08d6
fix: normalize endpoint display_name values to consistent convention (#20791)
Apply `{Provider} {Endpoint} API` naming convention to all endpoint
display names in provider_endpoints_support.json.

Ref: https://github.com/fastrepl/contextlengthof/issues/11
2026-02-12 20:31:35 -08:00
Cesar Garcia
0829000fca
feat: add 30 missing models to pricing JSON (#20797)
* feat: add 28 missing models to pricing JSON

Add models active in OpenAI and Gemini APIs but missing from the JSON.
All verified via litellm library and direct API calls.

OpenAI (15):
- TTS: tts-1-1106, tts-1-hd-1106, gpt-4o-mini-tts-2025-{03-20,12-15}
- Transcription: gpt-4o-mini-transcribe-2025-{03-20,12-15}
- Search: gpt-5-search-api, gpt-5-search-api-2025-10-14
- Realtime: gpt-realtime-mini-2025-{10-06,12-15}
- Video: sora-2, sora-2-pro
- Image: chatgpt-image-latest

Gemini (13):
- Image: gemini-2.0-flash-exp-image-generation (direct + gemini/ prefix)
- Chat: gemini/gemini-2.0-flash-lite-001
- Native Audio: gemini-2.5-flash-native-audio-{latest,preview-09-2025,preview-12-2025}
- TTS: gemini-2.5-flash-preview-tts
- Aliases: gemini-flash-latest, gemini-flash-lite-latest, gemini-pro-latest, gemini-exp-1206

* fix: correct pricing for realtime-mini models and add sora-2-pro-high-res

- Add missing cache_read_input_token_cost ($0.06/1M) to gpt-realtime-mini dated versions
- Add missing input_cost_per_image ($0.80/1M) to gpt-realtime-mini dated versions
- Add sora-2-pro-high-res ($0.50/sec) for 1024x1792/1792x1024 resolutions
- Add openai/sora-2-pro-high-res variant
2026-02-12 20:30:14 -08:00
Cesar Garcia
64d8f1a601
docs: add native thinking param examples for Claude Opus 4.6 (#20799)
* docs: add native thinking param examples for Claude Opus 4.6

Add documentation for using the native `thinking` parameter directly
with adaptive thinking and explicit budgets for Claude Opus 4.6.

* docs: add note about reasoning_effort mapping to adaptive for Opus 4.6
2026-02-12 20:29:28 -08:00
Sameer Kankute
19d4566bc8 Add encoder decoder for model id 2026-02-13 09:58:20 +05:30
Sameer Kankute
ad54d1e3ba Add target model names for vector store endpoints 2026-02-13 09:57:50 +05:30
datzscaler
f74fdfbb61
feat(ui): added UI for Zscaler AI Guard (#21077)
* fix: allow Management keys to access user/daily/activity and team/daily/activity

* feat(ui): added UI for Zscaler AI Guard

* feat(ui): addressed UI comment

* Apply suggestion from @greptile-apps[bot]

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: naaa760 <neh6a683@gmail.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-12 20:27:44 -08:00
Sameer Kankute
fa48166b10 Add _PROXY_LiteLLMManagedVectorStores class 2026-02-13 09:57:17 +05:30
Sameer Kankute
b2fd80869c Add BaseManagedResource class 2026-02-13 09:53:40 +05:30
jquinter
0a2f74b93e
fix: remove unused MCP_NPM_CACHE_DIR import from module level (#21084)
The constant is already imported locally at line 899 where it's actually used.
The top-level import was redundant and triggered ruff F401.
2026-02-12 20:19:46 -08:00
mubashir1osmani
1bc90d2db7
fix guardrail status error (#20972)
* fix guardrail status error

* fix function imports
2026-02-12 20:19:13 -08:00
pb
75e69e8d3b
feat(bedrock): extend model support (#21035)
* add nvidia.nemotron-nano-3-30b

* add zai.glm-4.7

* entend video input support for kimik2.5
2026-02-12 20:11:00 -08:00
joaokopernico
da31dd19da
fix(anthropic): use Authorization Bearer for OAuth tokens instead of x-api-key (#21039)
OAuth tokens (sk-ant-oat*) require Authorization: Bearer header per
Anthropic's OAuth specification, but were being sent via x-api-key
which Anthropic rejects with 'invalid x-api-key'.

- optionally_handle_anthropic_oauth: detect OAuth tokens in api_key
  param (standard chat flow), not just Authorization header
- get_anthropic_headers: use Authorization: Bearer + required OAuth
  headers for OAuth tokens, x-api-key for regular API keys
- Passthrough messages: skip x-api-key when Authorization is set
- Add oauth-2025-04-20 to beta headers whitelist config
2026-02-12 20:10:08 -08:00
Emerson Gomes
022846baae
fix(router): remove repeated provider parsing in budget limiter hot path (#21043)
* fix(router): remove budget limiter provider hot-path overhead

- avoid LiteLLM_Params instantiation from dict deployments in provider lookup\n- resolve provider once per deployment and reuse in budget filtering\n- add router unit tests for hot-path behavior\n\nFixes #21042

* fix(router): handle None provider cache entries in budget limiter

- avoid recomputing provider when cached value is None\n- clarify deployment_provider_map uses id(deployment) keys\n- add regression test covering None-provider cache path

* refactor(router): avoid id()-based provider cache coupling

- switch provider cache handoff to index-aligned list between budget-limiter loops\n- remove implicit dependency on object identity stability\n- move hot-path tests to tests/test_litellm/router_strategy per template guidance

* chore(router): make use_litellm_proxy default explicit

Use deployment_litellm_params.get('use_litellm_proxy', False) for clarity and parity with LiteLLM_Params default behavior.

* test(router): add provider-resolution parity guard

- wrap dict litellm_params with lightweight attribute view for get_llm_provider\n- reduce drift risk from manual field extraction vs LiteLLM_Params defaults\n- add parity test matrix comparing optimized path to legacy LiteLLM_Params behavior for dict deployments

* test(router): harden dict view compatibility for provider resolution

- extend _LiteLLMParamsDictView with mapping-like and dump methods\n- add regression test simulating future get_llm_provider method-based access\n- keep hot-path optimization while reducing duck-typing break risk

---------

Co-authored-by: Codex <codex@example.com>
2026-02-12 20:05:55 -08:00
Shivam Rawat
60390df4e2
Merge pull request #21083 from BerriAI/litellm_docs_clarity_on_dashscope
docs: add API base URLs for Dashscope (International and China/Beijing)
2026-02-12 18:47:38 -08:00
shivam
3c12f6f896 minor change 2026-02-12 18:44:16 -08:00
shivam
d4aa41daf6 docs: add API base URLs for Dashscope (International and China/Beijing) 2026-02-12 18:40:45 -08:00
yuneng-jiang
d701cc6588
Merge pull request #21081 from BerriAI/litellm_yj_release_changes_feb12
[Infra] CI/CD Fixes - Nightly Release Feb 12
2026-02-12 18:07:34 -08:00
Ishaan Jaff
9202e67e33
feat: MCP server discovery UI (#21079)
* feat: add curated MCP server registry for discovery UI

Curated list of 31 well-known MCP servers with names, icons,
categories, transport config, and registry URLs. Includes HTTP
endpoints for GitHub, Atlassian, Sentry, Snowflake, and Cloudflare.

* feat: add GET /v1/mcp/discover endpoint for MCP discovery

Admin-only endpoint that serves the curated MCP registry with
optional query and category filters. Used by the UI discovery modal.

* feat: add DiscoverableMCPServer types for MCP discovery

* feat: add fetchDiscoverableMCPServers network function

* feat: add MCP discovery modal component

Compact list-row layout with category filters, search, and
grouped server list. Follows dev-tool aesthetic.

* feat: wire MCP discovery modal into server management page

Add MCP Server button now opens discovery modal. Card click
pre-fills the create form. Custom Server opens blank form.

* feat: add prefill from discovery and back-to-registry link

Create form accepts prefillData from discovery selection and
shows a Browse MCP Registry link to return to discovery modal.

* test: add unit tests for MCP discovery endpoint and registry

Tests for registry JSON structure validation and endpoint
query/category filtering logic. 15 tests total.

* fix: sync registry with official MCP API and fix stdio prefill

- Updated transport types and URLs from registry.modelcontextprotocol.io API
- GitHub: streamable-http at api.githubcopilot.com/mcp/
- GitLab: streamable-http at gitlab.com/api/v4/mcp (remote only)
- Atlassian: SSE at mcp.atlassian.com/v1/sse (remote only)
- Linear: SSE at mcp.linear.app/sse (remote only)
- Notion: SSE at mcp.notion.com/sse (remote only)
- Stripe: streamable-http at mcp.stripe.com (remote only)
- Exa: streamable-http at mcp.exa.ai/mcp (remote only)
- Cloudflare: SSE at bindings.mcp.cloudflare.com/sse (remote only)
- Sentry: stdio via @sentry/mcp-server (npm, correct package)
- Snowflake: stdio via snowflake-labs-mcp (pypi/uvx, not npm)
- Brave Search: stdio via @brave/brave-search-mcp-server (correct package)
- Fixed stdio prefill to generate stdio_config JSON instead of separate fields
- Discovery modal matches create modal width and header style
- Back arrow positioned on left of create modal header

* Update ui/litellm-dashboard/src/components/mcp_tools/mcp_discovery.tsx

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update litellm/proxy/management_endpoints/mcp_management_endpoints.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: address Greptile review feedback

- Move `import json` and `import os` to module top level
- Move mcp_registry.json into litellm/proxy/ for pip distribution
- Fix `Text` component: destructure from antd Typography instead of deprecated Tremor
- Update test fixture path to match new registry location

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-12 17:59:21 -08:00
yuneng-jiang
3f8a1cf2dc bump: version 1.81.10 → 1.81.11 2026-02-12 17:55:01 -08:00
yuneng-jiang
111593397a fixing core proxy tests 2026-02-12 17:54:32 -08:00
yuneng-jiang
8d10311b4b content filter test fix 2026-02-12 17:54:16 -08:00
yuneng-jiang
e49d094606 fix openai tests 2026-02-12 17:53:47 -08:00
yuneng-jiang
c37e3be933 fixing mcp tests 2026-02-12 17:53:40 -08:00
yuneng-jiang
c41459c8e3 fixing mistral model deprecation, cloud zero transform bug 2026-02-12 17:53:04 -08:00
yuneng-jiang
514645777b adding key envs to docs 2026-02-12 17:52:35 -08:00
yuneng-jiang
0efe3ea825 bumping pillow and cryptography for security fixes 2026-02-12 17:52:20 -08:00
yuneng-jiang
2a4066646a fix mypy linting 2026-02-12 17:52:12 -08:00
yuneng-jiang
2c8225b465 fix ruff check 2026-02-12 17:52:00 -08:00
yuneng-jiang
918376ddaf
Merge pull request #20124 from naaa760/fix/management-key-access
fix: allow Management keys to access user/daily/activity and team
2026-02-12 17:11:18 -08:00
Harshit Jain
f77fbefc22
fix: resolve conflicts with verification e2e 2026-02-13 06:11:03 +05:30
milan-berri
a2e9e73b64
fix(proxy): change model mismatch logs from WARNING to DEBUG (#20994)
Fixes #20990

PR #19943 added logging when the proxy overrides model names to prevent
internal provider prefixes from leaking to clients. The behavior works
correctly but logs a WARNING on every request with model mismatch.

For high-traffic customers using model aliases or provider prefixes,
this creates millions of warnings per day, flooding logs and causing
disk space issues.

Changed log level from WARNING to DEBUG since:
- The model mismatch is expected behavior when using aliases
- The override happens correctly regardless of log level
- Operators can still enable with LITELLM_LOG=DEBUG for debugging

Changes:
- common_request_processing.py: 2 warnings -> debug (non-streaming)
- proxy_server.py: 1 warning -> debug (streaming)
2026-02-12 16:40:58 -08:00
yuneng-jiang
ce3bb97d40
Merge pull request #21076 from BerriAI/litellm_ui_model_table_cred
[Feature] UI - Model Page: Improve Credentials Messaging
2026-02-12 16:28:45 -08:00
Emerson Gomes
d9606773ea
feat(vertex_ai): add zai-org/glm-5-maas model pricing (#21053)
Add Vertex AI ZAI GLM-5 model map entry with reasoning + prompt caching metadata and cache-read pricing.\n\nRefs #21052

Co-authored-by: Codex <codex@example.com>
2026-02-12 16:11:59 -08:00
yuneng-jiang
168a8731ce improve credentials messaging 2026-02-12 16:02:54 -08:00
yuneng-jiang
3cbb12b9c8
Merge pull request #21074 from milan-berri/fix/mcp-server-name-validation-spaces
fix(ui): Block spaces and hyphens in MCP server names and aliases
2026-02-12 15:35:18 -08:00