Commit graph

46529 commits

Author SHA1 Message Date
yujonglee
352789257d
perf(rust): use pythonize for bridge serialization (#38764)
* perf(rust): use pythonize for bridge serialization

* perf(rust): benchmark serialization payload sizes

* refactor(rust): remove mutable benchmark state
2026-08-29 10:41:10 -07:00
ryan-crabbe-berri
9beb5ead4d fix(proxy): keep the exact model= DB predicate within the type-discipline budget
The where clause now uses the exact name string directly and skips the DB query when the typed search cannot occur in that name, so no new mutable literals are added (LIT002 gate).
2026-08-29 10:40:39 -07:00
ryan-crabbe-berri
e585aab3ea
Merge pull request #38771 from BerriAI/litellm_/api-reference-dark-mode-4ec67f
fix(ui): make code blocks follow the theme in dark mode
2026-08-29 10:39:25 -07:00
ryan-crabbe-berri
3e99ee8d0e fix(proxy): scope the DB-side model search by the exact model= filter
With model=<group>&search=<term>, the router list was narrowed to the group but the DB query only matched the substring, so other groups' rows leaked into the page and total_count.
2026-08-29 10:32:29 -07:00
ryan-crabbe-berri
4411562a0f fix(ui): satisfy lint and the modelInfoCall arity in the models hook tests 2026-08-29 10:32:29 -07:00
Yuneng Jiang
8f07a12726
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/api-reference-dark-mode-4ec67f 2026-08-29 10:25:55 -07:00
devin-ai-integration[bot]
30efcfd684
feat(mcp): support asymmetric (RS256) signing for MCP gateway session tokens (#38728)
* feat(mcp): support RS256 signing for MCP gateway session tokens

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style: ruff format session token modules

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): enforce key strength on rotated public keys and unique kids

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 10:18:59 -07:00
ryan-crabbe-berri
fe63ebdb19 fix(ui): filter the models page by exact model group instead of substring search
Pass the selected group as the exact model= param on /v2/model/info rather than as the substring search, so a group like gpt-4 no longer pulls gpt-4o rows into the page and count. Drop two comments that restated helper behavior.
2026-08-29 10:18:06 -07:00
devin-ai-integration[bot]
0de1825450
fix(health): honor allow_requests_on_db_unavailable in readiness probe (#37640)
* fix(health): honor allow_requests_on_db_unavailable in readiness probe

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(health): bound readiness DB check and pass reconnect timeout

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(health): bound whole readiness DB check with one deadline

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(health): keep readiness deadline fallback within lint budgets

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(health): suppress TQ008 for proxy-global readiness patches

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(db): release reconnect lock when a waiting reconnect is cancelled

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
2026-08-29 10:17:12 -07:00
Yuneng Jiang
03f64200c2
fix(ui): stop CodeBlock painting a hardcoded light background in dark mode
The customStyle background only reaches the <pre>, so the prism theme's own
background stayed on the inner <code>. In dark mode that left a white card
with a dark box hugging every line. Let the block inherit the themed surface
from its wrapper instead.
2026-08-29 10:15:05 -07:00
Mateo Wang
cb7d41a5c6
Merge pull request #38739 from BerriAI/litellm_fix_tag_routing_reads_merged_metadata_tags
fix(proxy): tag routing misses proxy-merged tags when chat requests carry litellm_metadata
2026-08-29 10:14:05 -07:00
Mateo Wang
ed2f1836de
Merge pull request #38738 from BerriAI/litellm_fix_batch_list_page_past_unparseable_rows
fix(batches): fill a managed batch page past rows that will not parse
2026-08-29 10:13:29 -07:00
yucheng-berri
f0fadb7f99
test(e2e): add logging e2e coverage (s3_v2, gcs_bucket, team langfuse callback, datadog failure) (#38552)
* test: add logging e2e coverage (s3_v2, gcs_bucket, team langfuse callback, datadog failure)

Five new live e2e scenarios raising Logging & Guardrails registry coverage:
s3_v2 success and failure objects read back from the real S3 bucket,
gcs_bucket success record read back through the GCS JSON API (with
nextPageToken pagination and per-request bearer minting), team-scoped
Langfuse callback delivery with non-team isolation, and DataDog failure
event delivery queried by indexed model_group. datadog_reader gains
query-based variants of the marker search; the langfuse cell is a new
registry row. Bucket readers settle past a full flush interval so a
late duplicate cannot hide from the exactly-one assertions

* test: cover clock-skew day prefix in gcs read-back and retry team callback propagation

* test: key the s3 failure read-back on the provider error, not payload absence

* chore: rerun ci

* chore: rerun ci after config sync

* chore: rerun ci with pr lane env

* chore: rerun ci

* chore: rerun ci

* chore: rerun ci

* chore: rerun ci

* chore: rerun ci

* test: add guardrail e2e coverage (presidio masking, bedrock post and during call, moderation on messages) (#38553)

* test: add guardrail e2e coverage (presidio masking, bedrock post/during, moderation on messages)

* test: require the phone placeholder positively in the presidio masking predicate

* test: count only the 400 verdict body as a bedrock post_call block

* test(e2e): exempt the guardrail config echo from the post_call leak assertion

* test(e2e): pin the fail-closed contract for an unknown guardrail name (skipped, product gap)

* test(e2e): tolerate the readiness 503 from a transient db blip in the callback-config probes
2026-08-29 09:43:44 -07:00
Rāna(Bass Ver.)
4f630411f0
Merge branch 'litellm_internal_staging' into fix/34379-unblock-customer 2026-08-30 00:17:26 +08:00
Mateo Wang
b2fe72e70c
Merge pull request #35940 from ump45nose/fix_databricks_oauth_url
fix(databricks): derive OAuth token URL from workspace origin
2026-08-29 07:30:14 -07:00
mateo-berri
c09f76a079 refactor(mcp): drop always-true isinstance guard on typed token-endpoint response 2026-08-29 07:15:00 -07:00
Mateo Wang
c42ac262d3
Merge branch 'litellm_internal_staging' into fix_databricks_oauth_url 2026-08-29 07:00:58 -07:00
mateo-berri
c4663a4ae9 chore(budgets): drop stale PLW0133 entry graduated to ruff.toml 2026-08-29 06:51:54 -07:00
mateo-berri
683330e4e8 merge(litellm_internal_staging): resolve typing sweep overlap, keep best-of-both types 2026-08-29 06:48:36 -07:00
mateo-berri
9c653be72c chore(budgets): drop merge headroom above staging ceilings 2026-08-29 06:42:59 -07:00
mateo-berri
163c7033d6 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5 2026-08-29 06:38:33 -07:00
mateo-berri
c37260a2bd Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5
# Conflicts:
#	basedpyright-code-budget.json
#	enterprise/litellm_enterprise/proxy/audit_logging_endpoints.py
#	litellm/_lazy_imports.py
#	litellm/a2a_protocol/litellm_completion_bridge/transformation.py
#	litellm/integrations/bitbucket/bitbucket_client.py
#	litellm/integrations/compression_interception/handler.py
#	litellm/integrations/prometheus_helpers/prometheus_api.py
#	litellm/litellm_core_utils/model_response_utils.py
#	litellm/litellm_core_utils/url_utils.py
#	litellm/llms/anthropic/experimental_pass_through/context_management/dispatcher.py
#	litellm/llms/anthropic/experimental_pass_through/responses_adapters/handler.py
#	litellm/llms/anthropic/skills/transformation.py
#	litellm/llms/azure/files/handler.py
#	litellm/llms/bedrock/realtime/handler.py
#	litellm/llms/chatgpt/chat/streaming_utils.py
#	litellm/llms/compactifai/chat/transformation.py
#	litellm/llms/oci/chat/cohere.py
#	litellm/llms/vertex_ai/vector_stores/rag_api/transformation.py
#	litellm/proxy/agent_endpoints/agent_registry.py
#	litellm/proxy/client/cli/commands/credentials.py
#	litellm/proxy/client/cli/commands/teams.py
#	litellm/proxy/common_utils/get_routes.py
#	litellm/proxy/db/routing_prisma_wrapper.py
#	litellm/proxy/guardrails/guardrail_hooks/custom_code/sandbox.py
#	litellm/proxy/guardrails/guardrail_hooks/hiddenlayer/hiddenlayer.py
#	litellm/proxy/guardrails/guardrail_hooks/llm_as_a_judge/__init__.py
#	litellm/proxy/guardrails/guardrail_hooks/promptguard/promptguard.py
#	litellm/rust_bridge/responses_websocket.py
#	litellm/secret_managers/secret_manager_handler.py
#	ruff-strict-budget.json
#	type-discipline-budget.json
2026-08-29 06:37:10 -07:00
Mateo Wang
e48f8f016f
Merge pull request #38148 from mubashir1osmani/litellm_hosted_vllm_videos
feat(hosted_vllm): add vLLM-Omni videos API
2026-08-29 06:30:35 -07:00
Devin AI
f0849eb0c9 fix(models): xai retirement repricing, bedrock grok-4.6 caching, openai/gemini deprecation dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 13:11:26 +00:00
mateo-berri
8d4620649f Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5
# Conflicts:
#	basedpyright-code-budget.json
#	litellm/caching/valkey_semantic_cache.py
#	litellm/integrations/compression_interception/handler.py
#	litellm/integrations/custom_logger.py
#	litellm/llms/custom_httpx/container_handler.py
#	litellm/llms/infinity/rerank/transformation.py
#	litellm/proxy/agent_endpoints/agent_registry.py
#	litellm/repositories/base_repository.py
#	litellm/repositories/credentials_repository.py
#	litellm/repositories/team_repository.py
#	ruff-strict-budget.json
#	type-discipline-budget.json
2026-08-29 06:03:33 -07:00
Devin AI
401b12e64b Merge remote-tracking branch 'origin/litellm_internal_staging' into devin/1787944648-registry-audit-rolling 2026-08-29 13:02:37 +00:00
Devin AI
bc2b7281e6 chore: ratchet basedpyright reportAny ceiling after merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 11:35:21 +00:00
Devin AI
adf5637fdf chore: merge litellm_internal_staging into litellm_techdebt_20260829
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 11:31:44 +00:00
Mateo Wang
ae2e23bb53
Merge pull request #38501 from BerriAI/litellm_decrease_anys_opus5_0826
refactor(types): replace Any with real types across 178 backend files
2026-08-29 04:29:44 -07:00
mateo-berri
ee86b62b66 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5_0826
# Conflicts:
#	basedpyright-code-budget.json
#	type-discipline-budget.json
2026-08-29 04:18:02 -07:00
Mateo Wang
6a3333d3c8
Merge pull request #38747 from BerriAI/litellm_aws_partition_helper
fix(aws): build every AWS endpoint and ARN from the region's partition (aws-cn, aws-us-gov)
2026-08-29 04:10:59 -07:00
Mateo Wang
39e4f1ae13
Merge pull request #38727 from BerriAI/litellm_aws_external_id_embed_sagemaker
fix(aws): forward aws_external_id in Bedrock embeddings and SageMaker credential loading
2026-08-29 03:30:32 -07:00
mateo-berri
47d8ce6d10 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5_0826
# Conflicts:
#	basedpyright-code-budget.json
#	litellm/llms/soniox/common_utils.py
#	ruff-strict-budget.json
#	type-discipline-budget.json
2026-08-29 03:29:39 -07:00
Mateo Wang
1fbf34e559
Merge pull request #38744 from BerriAI/litellm_fix_bedrock_batch_zero_count_retire
fix(bedrock): map real batch record counts and guard zero-count retire
2026-08-29 03:12:49 -07:00
Devin AI
e535724923 test: cover the bounded Hugging Face config fetch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 10:10:03 +00:00
Mateo Wang
efe51daf5b
Merge pull request #38742 from BerriAI/litellm_batch_id_fallback_pin
fix(router): pin batch, file, and fine-tuning job ids to their owning model group on fallback
2026-08-29 02:57:07 -07:00
Devin AI
1687c65823 fix: hardcode HF config fetch timeout instead of reading an env var
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 09:53:43 +00:00
Devin AI
db1b1e2195 fix: bound Hugging Face config fetch and keep embedding tests off the network
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 09:45:52 +00:00
mateo-berri
850b18d214 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_tag_routing_reads_merged_metadata_tags
# Conflicts:
#	tests/openai_endpoints_tests/test_e2e_openai_responses_api.py
2026-08-29 02:34:51 -07:00
mateo-berri
229970c500 fix(guardrails): unwrap HiddenParamsAsyncIteratorWrapper before deferred dispatch class sniffing 2026-08-29 02:30:34 -07:00
mateo-berri
1947c65081 test(aws): type the new partition test parameters 2026-08-29 02:27:18 -07:00
mateo-berri
664697133b Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_bedrock_guardrail_stream_audit 2026-08-29 02:24:59 -07:00
mateo-berri
f60ccf6234 fix(guardrails): match deferred stream dispatch shape per stream owner and defer passthrough logging until guardrail eos 2026-08-29 02:23:12 -07:00
mateo-berri
d5b5aca498 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_bedrock_batch_zero_count_retire 2026-08-29 02:21:28 -07:00
mateo-berri
2d11870793 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_batch_id_fallback_pin
# Conflicts:
#	tests/openai_endpoints_tests/test_e2e_openai_responses_api.py
2026-08-29 02:19:45 -07:00
mateo-berri
cd5cf84aac Merge branch 'litellm_internal_staging' into litellm_aws_partition_helper 2026-08-29 02:18:16 -07:00
Mateo Wang
3993829a21
Merge pull request #38748 from BerriAI/litellm_fix_gpt55_extra_body_test
test(responses): adapt temperature tests to the gpt-5 reasoning validation
2026-08-29 02:17:01 -07:00
mateo-berri
b0d485568b test(batches): pin the poller against retiring zero-count completed batches 2026-08-29 02:00:11 -07:00
mateo-berri
66947fcf7c test(responses): expect the bad-temperature 400 on a non-reasoning model 2026-08-29 01:58:15 -07:00
mateo-berri
8cf090b368 fix(router): arm the provider-scoped fallback pin only on resource-operating handlers 2026-08-29 01:46:33 -07:00