Commit graph

40072 commits

Author SHA1 Message Date
harish-berri
c8fb77f119
Merge pull request #25980 from BerriAI/litellm_metrics_auth
Fix /metrics hang when require_auth_for_metrics_endpoint is true and auth succeeds
2026-05-01 11:01:39 -07:00
user
7bda5c7cac chore(caching): isolate semantic cache entries 2026-05-01 10:59:49 -07:00
Yuneng Jiang
650821b538
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix-config-update-targeted-upserts
# Conflicts:
#	tests/test_litellm/proxy/test_proxy_server.py
2026-05-01 10:38:34 -07:00
Sameer Kankute
f88372919c
fix(mcp): survive Redis JSON round-trip for cached org object_permission
Raw Prisma model objects serialise to JSON on cache write but deserialise
as plain dicts on read (Redis backend). Attribute-style access on a dict
raises AttributeError, silently breaking org MCP permission enforcement.

Fix: convert the Prisma result to LiteLLM_ObjectPermissionTable (Pydantic)
before writing to cache using .dict(), and reconstruct the Pydantic model
from the cached dict on read — matching the pattern used by get_end_user_object
and get_team_object in auth_checks.py.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-01 23:06:38 +05:30
Sameer Kankute
4523f6af3d
fix(embeddings): allow omitting encoding_format via env sentinel none
Greptile: LITELLM_DEFAULT_EMBEDDING_ENCODING_FORMAT=none (case-insensitive)
strips encoding_format so OpenAI-compatible backends can use provider defaults.
Preserves optional_params passthrough when env is unset.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-01 23:03:13 +05:30
Cursor Agent
7deac91a92
Fix Vertex batch custom id labels 2026-05-01 17:29:50 +00:00
Sameer Kankute
8300657af9
fix(mcp): preserve oauth2 m2m auth for tools routes (#26871)
* Fix tool/list M2M creds issue

* Fix tool call creds issue

* Fix greptile review

* Fix lint

* Fix lint

* Fix lint

* Fix lint
2026-05-01 10:26:10 -07:00
Cursor Agent
97cfb4f6fd
Unify secret redaction patterns 2026-05-01 17:24:21 +00:00
Mateo Wang
a155ea1e8a
Merge pull request #26076 from BerriAI/litellm_vertex_model_garden_xai_openapi
feat(vertex_ai): Model Garden OpenAPI for publisher model ids
2026-05-01 10:15:58 -07:00
harish-berri
32ab390e7e
Merge pull request #26202 from BerriAI/litellm_token_verification_query_opt
Litellm token verification query optimization
2026-05-01 10:10:07 -07:00
Mateo Wang
026ee8839c
Merge pull request #25270 from BerriAI/litellm_oss_staging_04_06_2026 2026-05-01 08:04:25 -07:00
Cursor Agent
fca21a979c
Fix org MCP permission ceiling escalation 2026-05-01 14:54:29 +00:00
Cursor Agent
df41973110
Remove dead embedding encoding fallback 2026-05-01 14:51:19 +00:00
Mateo Wang
6552e3c5c0
Merge pull request #26402 from BerriAI/litellm_azure-container-file-routing-fix 2026-05-01 07:50:37 -07:00
Mateo Wang
dddbfd54a1
Merge pull request #26950 from BerriAI/litellm_batch_model_id_mapping 2026-05-01 07:47:47 -07:00
Sameer Kankute
8b85deb1df
Merge pull request #26985 from BerriAI/litellm_internal_staging
merge main
2026-05-01 19:07:11 +05:30
Sameer Kankute
a94ae62706
Merge pull request #26984 from BerriAI/litellm_internal_staging
merge main
2026-05-01 19:03:59 +05:30
Sameer Kankute
e942ef6eea
Fix black 2026-05-01 18:30:33 +05:30
Sameer Kankute
0a9c076e8b
Fix greptile review 2026-05-01 18:28:24 +05:30
Sameer Kankute
493405c129
Fix lint 2026-05-01 18:22:53 +05:30
Sameer Kankute
8d9db7b4a6
fix(gemini): follow provider defaults for Gemini 3 thinking
Some checks failed
Unit Tests: Caching (Redis) / caching-redis (push) Has been cancelled
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Has been cancelled
Unit Tests: Security / security (push) Has been cancelled
Unit Tests: Proxy DB Operations / auth-checks (push) Has been cancelled
Unit Tests: Proxy DB Operations / budgets (push) Has been cancelled
Unit Tests: Proxy DB Operations / custom-logging (push) Has been cancelled
Unit Tests: Proxy DB Operations / db-and-spend (push) Has been cancelled
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Has been cancelled
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Has been cancelled
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Has been cancelled
Unit Tests: Proxy DB Operations / key-generation (push) Has been cancelled
Unit Tests: Proxy DB Operations / logging-misc (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-runtime (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-server-core (push) Has been cancelled
Unit Tests: Proxy DB Operations / schema-migration (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-utils (push) Has been cancelled
Stop forcing Gemini 3 thinkingLevel for Anthropic-style thinking params by default, and gate legacy low/minimal mapping behind an explicit feature flag to avoid provider-default confusion.

Made-with: Cursor
2026-05-01 18:14:21 +05:30
Sameer Kankute
e656b2a47b
correct model map 2026-05-01 18:07:33 +05:30
Sameer Kankute
71e2b6fcad
Fix reviews 2026-05-01 18:05:08 +05:30
Sameer Kankute
19813527fa
feat(vertex_ai): Model Garden OpenAPI for publisher model ids
- Route publisher/model ids (e.g. xai/grok) to .../endpoints/openapi; keep model in JSON body
- Add model_prices keys for vertex_ai/openai/xai/grok-*
- Document xAI Grok on vertex_partner (aligned with GPT-OSS)
- Add tests for create_vertex_url and body-model heuristic

Made-with: Cursor
2026-05-01 18:05:08 +05:30
Sameer Kankute
5feb6008d8
Fix greptile review 2026-05-01 17:56:37 +05:30
Sameer Kankute
475220ecc6
Fix greptile review 2026-05-01 17:46:37 +05:30
Sameer Kankute
3eccededcc
Fix black 2026-05-01 17:33:50 +05:30
Sameer Kankute
99d075d863
fix code qa 2026-05-01 17:27:52 +05:30
Emmanuel Acheampong
f8ba2d750b
fix(crusoe): fix streaming doc model typo and add supports_vision for Gemma 3
- Streaming example referenced Llama-3.1 instead of Llama-3.3
- Add supports_vision: true for gemma-3-12b-it in both JSON files,
  matching other providers (bedrock, novita)
2026-05-01 17:27:52 +05:30
Emmanuel Acheampong
e08b8ef7b6
fix(crusoe): split Custom API Base docs into two independent examples
The previous example set CRUSOE_API_BASE via env var and also passed
api_base= in the same call, making it look like both were required.
They are independent alternatives.
2026-05-01 17:27:52 +05:30
Emmanuel Acheampong
6e1e6244cf
fix(crusoe): remove trailing slashes from API base URLs and fix list indentation
Trailing slashes on custom API base examples cause double-slash in
get_complete_url. Also fixes inconsistent list indentation in
test_crusoe_models_configuration.
2026-05-01 17:27:52 +05:30
Emmanuel Acheampong
2805572e9b
fix(crusoe): add param_mappings for max_completion_tokens → max_tokens
Crusoe's vLLM-based endpoint accepts max_tokens, not max_completion_tokens.
Without this mapping, callers using the OpenAI-standard param would get errors.
2026-05-01 17:27:52 +05:30
Emmanuel Acheampong
51f8e5a57b
feat(crusoe): add supports_reasoning flag for DeepSeek-R1 and Kimi-K2-Thinking
These are reasoning/thinking models but were missing the flag, causing
litellm.supports_reasoning() to return False and reasoning-token handling
to not activate.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-01 17:27:52 +05:30
Emmanuel Acheampong
133512dcc6
fix(crusoe): sync backup model cost map with main file
The backup JSON was missing Crusoe model entries, causing
test_crusoe_model_list_populated to fail with AssertionError.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-01 17:27:52 +05:30
Emmanuel Acheampong
9039eb1898
fix(crusoe): fix docs trailing slash, test state pollution, missing __init__.py
- Remove trailing slash from docs Base URL to match providers.json
- Wrap model_cost mutations in try/finally to prevent test state leakage
- Add missing __init__.py to crusoe test package
2026-05-01 17:27:52 +05:30
Emmanuel Acheampong
d7313496f3
fix: remove trailing slash from CRUSOE_API_BASE and unused sys import 2026-05-01 17:27:52 +05:30
Krish Dholakia
9d64cc8ff8
Apply suggestion from @greptile-apps[bot]
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-05-01 17:27:52 +05:30
Emmanuel Acheampong
6ae7929d7c
Apply suggestion from @greptile-apps[bot]
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-05-01 17:27:52 +05:30
Emmanuel Acheampong
d492d8fe82
refactor(crusoe): simplify to JSON-based provider registration
Replace hand-written CrusoeChatConfig class and manual registrations
across constants.py, __init__.py, get_llm_provider_logic.py, and
_lazy_imports_registry.py with a single entry in
litellm/llms/openai_like/providers.json, consistent with the
recommended pattern for OpenAI-compatible providers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 17:27:52 +05:30
Emmanuel Acheampong
caa0db3843
adding crusoe to litellm 2026-05-01 17:27:34 +05:30
Sameer Kankute
8c55c541d3
Fix ruff errors 2026-05-01 17:27:34 +05:30
Sameer Kankute
319180f650
Fix ruff errors 2026-05-01 17:27:04 +05:30
Sameer Kankute
ed853e138f
Fix code qa 2026-05-01 17:26:46 +05:30
Sameer Kankute
a86938a4af
Fix import error 2026-05-01 17:26:46 +05:30
Sameer Kankute
b3fdb5cc69
Fix import error 2026-05-01 17:26:46 +05:30
Sameer Kankute
b0ac74c556
Fix import error 2026-05-01 17:26:46 +05:30
Yuneng Jiang
9b90fc07d2
chore: fixes 2026-05-01 17:26:46 +05:30
Sameer Kankute
b8f5189b65
fix(azure): forward api_version to aembedding() for Azure AI Foundry v1 endpoints (#24911)
When aembedding=True, api_version was not passed to self.aembedding(), causing
get_azure_openai_client() to receive None instead of "v1". This made
_is_azure_v1_api_version() return False, so AsyncAzureOpenAI was selected
instead of AsyncOpenAI, constructing the wrong request URL and returning 404.

Fixes #24848

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 17:26:46 +05:30
d 🔹
9d6983c4c0
fix(gemini): handle Gemini Files API URIs without fetching (#24922)
* fix(gemini): handle Gemini Files API URIs without fetching

Fixes #24907

When a file is uploaded via the Gemini Files API, the returned URI
(https://generativelanguage.googleapis.com/v1beta/files/...) starts
with 'https://' and hits the generic HTTPS handler in
_process_gemini_media(). That handler calls
_get_image_mime_type_from_url() which tries to fetch the URL — but
Gemini Files API URLs return 403 when accessed directly, causing:
  'Unable to determine mime type for file_id: ...'

Fix: add an early elif that matches Gemini Files API URLs and passes
them through as file_data without trying to fetch the URL. When an
explicit format is provided it's included; otherwise the Gemini API
infers the MIME type from its stored metadata.

Exactly matches the fix direction suggested by the issue reporter
(rodriciru).

* fix: anchor Gemini Files API URL check with startswith

Address greptile P2: replace `in` substring check with `startswith`
to prevent query-string injection bypass (e.g.
`https://evil.com/?ref=https://generativelanguage...`).

Also adds trailing slash to match only valid file URIs.

---------

Co-authored-by: voidborne-d <voidborne-d@users.noreply.github.com>
2026-05-01 17:26:45 +05:30
milan-berri
7e58c7139a
fix(proxy): include team membership budget in combined_view for RPM/TPM (#24925)
Join LiteLLM_BudgetTable as b_tm on team membership budget_id and select
team_member_tpm_limit / team_member_rpm_limit so virtual key auth populates
limits for parallel_request_limiter_v3.

Add test_team_member_rate_limits_v3_raises_429_when_over_limit mirroring
existing key-level OVER_LIMIT / HTTP 429 coverage.

Made-with: Cursor
2026-05-01 17:26:45 +05:30