Commit graph

45690 commits

Author SHA1 Message Date
Yuneng Jiang
8f07a12726
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/api-reference-dark-mode-4ec67f 2026-08-29 10:25:55 -07:00
devin-ai-integration[bot]
30efcfd684
feat(mcp): support asymmetric (RS256) signing for MCP gateway session tokens (#38728)
* feat(mcp): support RS256 signing for MCP gateway session tokens

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style: ruff format session token modules

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): enforce key strength on rotated public keys and unique kids

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 10:18:59 -07:00
ryan-crabbe-berri
fe63ebdb19 fix(ui): filter the models page by exact model group instead of substring search
Pass the selected group as the exact model= param on /v2/model/info rather than as the substring search, so a group like gpt-4 no longer pulls gpt-4o rows into the page and count. Drop two comments that restated helper behavior.
2026-08-29 10:18:06 -07:00
devin-ai-integration[bot]
0de1825450
fix(health): honor allow_requests_on_db_unavailable in readiness probe (#37640)
* fix(health): honor allow_requests_on_db_unavailable in readiness probe

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(health): bound readiness DB check and pass reconnect timeout

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(health): bound whole readiness DB check with one deadline

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(health): keep readiness deadline fallback within lint budgets

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(health): suppress TQ008 for proxy-global readiness patches

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(db): release reconnect lock when a waiting reconnect is cancelled

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
2026-08-29 10:17:12 -07:00
Yuneng Jiang
03f64200c2
fix(ui): stop CodeBlock painting a hardcoded light background in dark mode
The customStyle background only reaches the <pre>, so the prism theme's own
background stayed on the inner <code>. In dark mode that left a white card
with a dark box hugging every line. Let the block inherit the themed surface
from its wrapper instead.
2026-08-29 10:15:05 -07:00
Mateo Wang
cb7d41a5c6
Merge pull request #38739 from BerriAI/litellm_fix_tag_routing_reads_merged_metadata_tags
fix(proxy): tag routing misses proxy-merged tags when chat requests carry litellm_metadata
2026-08-29 10:14:05 -07:00
Mateo Wang
ed2f1836de
Merge pull request #38738 from BerriAI/litellm_fix_batch_list_page_past_unparseable_rows
fix(batches): fill a managed batch page past rows that will not parse
2026-08-29 10:13:29 -07:00
yucheng-berri
f0fadb7f99
test(e2e): add logging e2e coverage (s3_v2, gcs_bucket, team langfuse callback, datadog failure) (#38552)
* test: add logging e2e coverage (s3_v2, gcs_bucket, team langfuse callback, datadog failure)

Five new live e2e scenarios raising Logging & Guardrails registry coverage:
s3_v2 success and failure objects read back from the real S3 bucket,
gcs_bucket success record read back through the GCS JSON API (with
nextPageToken pagination and per-request bearer minting), team-scoped
Langfuse callback delivery with non-team isolation, and DataDog failure
event delivery queried by indexed model_group. datadog_reader gains
query-based variants of the marker search; the langfuse cell is a new
registry row. Bucket readers settle past a full flush interval so a
late duplicate cannot hide from the exactly-one assertions

* test: cover clock-skew day prefix in gcs read-back and retry team callback propagation

* test: key the s3 failure read-back on the provider error, not payload absence

* chore: rerun ci

* chore: rerun ci after config sync

* chore: rerun ci with pr lane env

* chore: rerun ci

* chore: rerun ci

* chore: rerun ci

* chore: rerun ci

* chore: rerun ci

* test: add guardrail e2e coverage (presidio masking, bedrock post and during call, moderation on messages) (#38553)

* test: add guardrail e2e coverage (presidio masking, bedrock post/during, moderation on messages)

* test: require the phone placeholder positively in the presidio masking predicate

* test: count only the 400 verdict body as a bedrock post_call block

* test(e2e): exempt the guardrail config echo from the post_call leak assertion

* test(e2e): pin the fail-closed contract for an unknown guardrail name (skipped, product gap)

* test(e2e): tolerate the readiness 503 from a transient db blip in the callback-config probes
2026-08-29 09:43:44 -07:00
Mateo Wang
b2fe72e70c
Merge pull request #35940 from ump45nose/fix_databricks_oauth_url
fix(databricks): derive OAuth token URL from workspace origin
2026-08-29 07:30:14 -07:00
Mateo Wang
c42ac262d3
Merge branch 'litellm_internal_staging' into fix_databricks_oauth_url 2026-08-29 07:00:58 -07:00
Mateo Wang
e48f8f016f
Merge pull request #38148 from mubashir1osmani/litellm_hosted_vllm_videos
feat(hosted_vllm): add vLLM-Omni videos API
2026-08-29 06:30:35 -07:00
Devin AI
f0849eb0c9 fix(models): xai retirement repricing, bedrock grok-4.6 caching, openai/gemini deprecation dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 13:11:26 +00:00
Devin AI
401b12e64b Merge remote-tracking branch 'origin/litellm_internal_staging' into devin/1787944648-registry-audit-rolling 2026-08-29 13:02:37 +00:00
Devin AI
bc2b7281e6 chore: ratchet basedpyright reportAny ceiling after merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 11:35:21 +00:00
Devin AI
adf5637fdf chore: merge litellm_internal_staging into litellm_techdebt_20260829
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 11:31:44 +00:00
Mateo Wang
ae2e23bb53
Merge pull request #38501 from BerriAI/litellm_decrease_anys_opus5_0826
refactor(types): replace Any with real types across 178 backend files
2026-08-29 04:29:44 -07:00
mateo-berri
ee86b62b66 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5_0826
# Conflicts:
#	basedpyright-code-budget.json
#	type-discipline-budget.json
2026-08-29 04:18:02 -07:00
Mateo Wang
6a3333d3c8
Merge pull request #38747 from BerriAI/litellm_aws_partition_helper
fix(aws): build every AWS endpoint and ARN from the region's partition (aws-cn, aws-us-gov)
2026-08-29 04:10:59 -07:00
Mateo Wang
39e4f1ae13
Merge pull request #38727 from BerriAI/litellm_aws_external_id_embed_sagemaker
fix(aws): forward aws_external_id in Bedrock embeddings and SageMaker credential loading
2026-08-29 03:30:32 -07:00
mateo-berri
47d8ce6d10 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5_0826
# Conflicts:
#	basedpyright-code-budget.json
#	litellm/llms/soniox/common_utils.py
#	ruff-strict-budget.json
#	type-discipline-budget.json
2026-08-29 03:29:39 -07:00
Mateo Wang
1fbf34e559
Merge pull request #38744 from BerriAI/litellm_fix_bedrock_batch_zero_count_retire
fix(bedrock): map real batch record counts and guard zero-count retire
2026-08-29 03:12:49 -07:00
Devin AI
e535724923 test: cover the bounded Hugging Face config fetch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 10:10:03 +00:00
Mateo Wang
efe51daf5b
Merge pull request #38742 from BerriAI/litellm_batch_id_fallback_pin
fix(router): pin batch, file, and fine-tuning job ids to their owning model group on fallback
2026-08-29 02:57:07 -07:00
Devin AI
1687c65823 fix: hardcode HF config fetch timeout instead of reading an env var
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 09:53:43 +00:00
Devin AI
db1b1e2195 fix: bound Hugging Face config fetch and keep embedding tests off the network
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 09:45:52 +00:00
mateo-berri
850b18d214 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_tag_routing_reads_merged_metadata_tags
# Conflicts:
#	tests/openai_endpoints_tests/test_e2e_openai_responses_api.py
2026-08-29 02:34:51 -07:00
mateo-berri
1947c65081 test(aws): type the new partition test parameters 2026-08-29 02:27:18 -07:00
mateo-berri
d5b5aca498 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_bedrock_batch_zero_count_retire 2026-08-29 02:21:28 -07:00
mateo-berri
2d11870793 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_batch_id_fallback_pin
# Conflicts:
#	tests/openai_endpoints_tests/test_e2e_openai_responses_api.py
2026-08-29 02:19:45 -07:00
mateo-berri
cd5cf84aac Merge branch 'litellm_internal_staging' into litellm_aws_partition_helper 2026-08-29 02:18:16 -07:00
Mateo Wang
3993829a21
Merge pull request #38748 from BerriAI/litellm_fix_gpt55_extra_body_test
test(responses): adapt temperature tests to the gpt-5 reasoning validation
2026-08-29 02:17:01 -07:00
mateo-berri
b0d485568b test(batches): pin the poller against retiring zero-count completed batches 2026-08-29 02:00:11 -07:00
mateo-berri
66947fcf7c test(responses): expect the bad-temperature 400 on a non-reasoning model 2026-08-29 01:58:15 -07:00
mateo-berri
8cf090b368 fix(router): arm the provider-scoped fallback pin only on resource-operating handlers 2026-08-29 01:46:33 -07:00
mateo-berri
7fbcd9c3ed fix(bedrock): treat partial record counts as unknown on batch retrieve 2026-08-29 01:40:07 -07:00
mateo-berri
81eb014e71 test(responses): use a reasoning-legal temperature in the gpt-5.5 extra_body merge test 2026-08-29 01:36:45 -07:00
mateo-berri
2968c246b9 test(responses): repair two tests broken by the gpt-5 temperature gate
PR #38593 stopped forwarding temperature to reasoning models, which left
test_extra_body_merges_with_request_data raising UnsupportedParamsError
and test_bad_request_bad_param_error no longer getting a rejection from
OpenAI because drop_params now eats the param. Both repairs are the same
hunks PR #38739 carries, so the branches merge clean in either order
2026-08-29 01:32:09 -07:00
mateo-berri
9da766a1f9 test: use a non-reasoning model for the responses bad-param e2e fixture 2026-08-29 01:22:49 -07:00
mateo-berri
ad8c1457d1 fix(aws): build every AWS endpoint and ARN from the region partition
Adds litellm/litellm_core_utils/aws_partition.py mapping a region to its
AWS partition (aws, aws-cn, aws-us-gov, and the iso partitions), its DNS
suffix, and its ARN prefix, and uses it at every AWS host and ARN build
site: bedrock (runtime, agent, agentcore, legacy client, batches, files,
realtime), sagemaker, polly, secrets manager, s3 log uploads, bedrock
passthrough routes, and rag ingestion. ARN detection now accepts
arn:aws-cn: and arn:aws-us-gov: prefixes.

STS region resolution now falls back to the configured aws_region_name
after the aws_sts_endpoint host and the AWS_REGION/AWS_DEFAULT_REGION env
vars, so cn and gov role assumption no longer silently signs against
us-west-2.

A partition sweep test walks every endpoint builder with cn regions and
asserts no amazonaws.com host or arn:aws: prefix comes out, plus an AST
guard that fails on any new f-string hardcoding either literal.
2026-08-29 01:21:59 -07:00
mateo-berri
1956903c28 refactor(vertex_ai): narrow dict return annotations in gemini transcribe config 2026-08-29 01:21:38 -07:00
mateo-berri
6301a520a6 test: pin reasoning effort none in the responses extra_body fixture 2026-08-29 01:07:24 -07:00
mateo-berri
5d34fb20ff fix(bedrock): map real batch record counts and guard zero-count retire 2026-08-29 01:05:36 -07:00
Devin AI
9bfb332904 refactor: replace fresh getattr/setattr and test type-ignores with typed access
Same-day debt cleanup on code that landed in the last 24 hours. No behavior change.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 07:59:27 +00:00
mateo-berri
8fcfaf519c test: drop duplicated callback assertion 2026-08-29 00:53:24 -07:00
mateo-berri
817b5f44f6 fix(anthropic_endpoints): serialize dict-detail HTTPExceptions on /v1/messages like sibling surfaces 2026-08-29 00:48:14 -07:00
mateo-berri
0b14897a7e fix(router): pin batch, file, and fine-tuning job ids to their owning model group on fallback 2026-08-29 00:47:34 -07:00
mateo-berri
1a26608769 feat(vertex_ai): route gemini transcribe models to generateContent on /v1/audio/transcriptions 2026-08-29 00:46:03 -07:00
mateo-berri
f58c3c0868 fix(proxy): fold litellm_metadata into metadata on chat routes so tag routing sees merged tags 2026-08-29 00:42:43 -07:00
mateo-berri
d7bd6ca614 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_aws_external_id_embed_sagemaker 2026-08-29 00:17:05 -07:00
Mateo Wang
ae7e50f096
Merge pull request #38607 from BerriAI/litellm_fix_files_pre_call_hook
fix(proxy): trigger async_pre_call_hook on POST /v1/files uploads
2026-08-28 23:58:05 -07:00