Commit graph

13916 commits

Author SHA1 Message Date
mateo-berri
f0c6efc695 fix(ci): skip variable-priced catalog rows so the schema gate stays green 2026-08-29 13:27:21 -07:00
mateo-berri
9c5db54d53 fix(ci): sync openrouter and vercel_ai_gateway prices without crashing on unpriced catalog rows 2026-08-29 13:22:05 -07:00
Mateo Wang
9ed7de6c02
Merge pull request #38670 from BerriAI/devin_ai_38659_cohere_embed_dispatch
fix(bedrock): route all cohere.embed models to the cohere embedding config
2026-08-29 12:55:32 -07:00
Mateo Wang
817bbe1dc6
Merge pull request #34440 from dan2k3k4/litellm_soniox_srt_cue_grouping
fix(soniox): align synthesized SRT/VTT cues to real speech timing
2026-08-29 12:49:57 -07:00
Mateo Wang
c453920f7a
Merge pull request #38285 from BerriAI/litellm_azure_v1_image_routes
fix(azure): use /openai/v1 image routes for v1, preview and latest api versions
2026-08-29 12:45:37 -07:00
mateo-berri
886d39c3a2 test(bedrock): expect cohere embed base64 encoding_format to normalize to float 2026-08-29 12:44:01 -07:00
yuneng-jiang
d435a62ce6
Merge pull request #38782 from BerriAI/litellm_/logs-reopen-shadcn-migration-2c9526
fix(ui): restore the reopen control for the log drawer's trace sidebar
2026-08-29 12:38:26 -07:00
mateo-berri
a3eac3f771 fix(bedrock): normalize encoding_format base64 to float for cohere embed models 2026-08-29 12:13:09 -07:00
mateo-berri
4c42c01cb2 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_soniox_srt_cue_grouping
# Conflicts:
#	litellm/llms/soniox/common_utils.py
2026-08-29 12:02:21 -07:00
Mateo Wang
002d0068f5
Merge pull request #38580 from BerriAI/devin_ai_fix_model_new_read_replica_lag_38556
fix(proxy): pin model reconcile read to the writer DB so /model/new does not 500 under read replica lag
2026-08-29 12:02:08 -07:00
Yuneng Jiang
08c83c12e9
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/logs-reopen-shadcn-migration-2c9526 2026-08-29 11:55:20 -07:00
Mateo Wang
c24f821652
Merge pull request #34849 from BerriAI/litellm_keyless_key_managed_resource_owner
fix(managed resources): let keys with no user_id or team_id read their own batches and files
2026-08-29 11:50:00 -07:00
Mateo Wang
5e60ec5c31
Merge pull request #38743 from BerriAI/litellm_techdebt_20260829
refactor: clean up tech debt that landed on 2026-08-29
2026-08-29 11:46:05 -07:00
mateo-berri
6d3e687ce4 fix(db): let the writer pin yield to the replica while the writer is degraded 2026-08-29 11:44:45 -07:00
mateo-berri
9e01bd1441 fix(azure): send the deployment name as the body model on v1 image routes 2026-08-29 11:41:58 -07:00
mateo-berri
68f891fd2b Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_keyless_key_managed_resource_owner 2026-08-29 11:33:08 -07:00
mateo-berri
a8c36e8307 Merge remote-tracking branch 'origin/litellm_internal_staging' into devin_ai_fix_model_new_read_replica_lag_38556 2026-08-29 11:29:07 -07:00
yuneng-jiang
fa25ff2a2e
Merge pull request #38626 from BerriAI/litellm_ui_model_links_team_key_info
feat(ui): link team and key model chips to the models page filtered to that group
2026-08-29 11:26:15 -07:00
tin-berri
2a5d09ee87
fix(policy): let the AI policy suggester drop sampling params its model refuses (#38594)
The suggester pins temperature=0.2 for tool-selection determinism and passed no
drop_params, so an operator-supplied reasoning model whose only accepted temperature is 1
made litellm raise UnsupportedParamsError and the whole suggestion fail. The default
gpt-4o-mini is unaffected; the failure needs the caller to name a model.

Every other internal LLM call the proxy makes on a user's behalf already opts in through
judge_acompletion, which sets drop_params=True on both dispatch paths. This was the one
caller outside that contract, so the sampling preference is now advisory here too and the
call degrades instead of dying.

Resolves LIT-6352
2026-08-29 10:58:27 -07:00
mateo-berri
acb621c35b Merge remote-tracking branch 'origin/litellm_internal_staging' into devin_ai_fix_model_new_read_replica_lag_38556 2026-08-29 10:54:46 -07:00
ryan-crabbe-berri
08118e6246 fix(proxy): let the exact model= filter match team BYOK public names
Team-scoped deployments keep the internal model_name_{team_id}_{uuid} routing key and expose the public name in model_info.team_public_model_name. The dashboard links team model chips with the public name, so the exact filter now matches either name via the existing helper.
2026-08-29 10:52:15 -07:00
Yuneng Jiang
8278a92a06
fix(ui): let the log drawer's trace sidebar expand again once collapsed
The sidebar toggle sits absolutely positioned over the drawer's flex row. With
the sidebar expanded it lands on the sidebar header, but once collapsed it lands
on the drawer header, which is sticky at z-chrome (10). The named-z-scale
refactor moved the toggle from z-20 to z-raised (1), so from then on the header
painted over it and swallowed the click: collapse the trace list and there was
no way to bring it back.

Moves the toggle to z-floating (30) and folds the two mirrored buttons into one,
since they only ever differed by icon, label and handler.

Covered by a Playwright spec, which is the tier that can see the layering: the
button stays visible and enabled either way, so the pre-fix failure is a click
interception that jsdom cannot reproduce.
2026-08-29 10:42:48 -07:00
ryan-crabbe-berri
9beb5ead4d fix(proxy): keep the exact model= DB predicate within the type-discipline budget
The where clause now uses the exact name string directly and skips the DB query when the typed search cannot occur in that name, so no new mutable literals are added (LIT002 gate).
2026-08-29 10:40:39 -07:00
ryan-crabbe-berri
3e99ee8d0e fix(proxy): scope the DB-side model search by the exact model= filter
With model=<group>&search=<term>, the router list was narrowed to the group but the DB query only matched the substring, so other groups' rows leaked into the page and total_count.
2026-08-29 10:32:29 -07:00
devin-ai-integration[bot]
30efcfd684
feat(mcp): support asymmetric (RS256) signing for MCP gateway session tokens (#38728)
* feat(mcp): support RS256 signing for MCP gateway session tokens

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style: ruff format session token modules

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): enforce key strength on rotated public keys and unique kids

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 10:18:59 -07:00
devin-ai-integration[bot]
0de1825450
fix(health): honor allow_requests_on_db_unavailable in readiness probe (#37640)
* fix(health): honor allow_requests_on_db_unavailable in readiness probe

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(health): bound readiness DB check and pass reconnect timeout

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(health): bound whole readiness DB check with one deadline

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(health): keep readiness deadline fallback within lint budgets

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(health): suppress TQ008 for proxy-global readiness patches

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(db): release reconnect lock when a waiting reconnect is cancelled

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
2026-08-29 10:17:12 -07:00
Mateo Wang
cb7d41a5c6
Merge pull request #38739 from BerriAI/litellm_fix_tag_routing_reads_merged_metadata_tags
fix(proxy): tag routing misses proxy-merged tags when chat requests carry litellm_metadata
2026-08-29 10:14:05 -07:00
Mateo Wang
ed2f1836de
Merge pull request #38738 from BerriAI/litellm_fix_batch_list_page_past_unparseable_rows
fix(batches): fill a managed batch page past rows that will not parse
2026-08-29 10:13:29 -07:00
yucheng-berri
f0fadb7f99
test(e2e): add logging e2e coverage (s3_v2, gcs_bucket, team langfuse callback, datadog failure) (#38552)
* test: add logging e2e coverage (s3_v2, gcs_bucket, team langfuse callback, datadog failure)

Five new live e2e scenarios raising Logging & Guardrails registry coverage:
s3_v2 success and failure objects read back from the real S3 bucket,
gcs_bucket success record read back through the GCS JSON API (with
nextPageToken pagination and per-request bearer minting), team-scoped
Langfuse callback delivery with non-team isolation, and DataDog failure
event delivery queried by indexed model_group. datadog_reader gains
query-based variants of the marker search; the langfuse cell is a new
registry row. Bucket readers settle past a full flush interval so a
late duplicate cannot hide from the exactly-one assertions

* test: cover clock-skew day prefix in gcs read-back and retry team callback propagation

* test: key the s3 failure read-back on the provider error, not payload absence

* chore: rerun ci

* chore: rerun ci after config sync

* chore: rerun ci with pr lane env

* chore: rerun ci

* chore: rerun ci

* chore: rerun ci

* chore: rerun ci

* chore: rerun ci

* test: add guardrail e2e coverage (presidio masking, bedrock post and during call, moderation on messages) (#38553)

* test: add guardrail e2e coverage (presidio masking, bedrock post/during, moderation on messages)

* test: require the phone placeholder positively in the presidio masking predicate

* test: count only the 400 verdict body as a bedrock post_call block

* test(e2e): exempt the guardrail config echo from the post_call leak assertion

* test(e2e): pin the fail-closed contract for an unknown guardrail name (skipped, product gap)

* test(e2e): tolerate the readiness 503 from a transient db blip in the callback-config probes
2026-08-29 09:43:44 -07:00
Mateo Wang
c42ac262d3
Merge branch 'litellm_internal_staging' into fix_databricks_oauth_url 2026-08-29 07:00:58 -07:00
Mateo Wang
e48f8f016f
Merge pull request #38148 from mubashir1osmani/litellm_hosted_vllm_videos
feat(hosted_vllm): add vLLM-Omni videos API
2026-08-29 06:30:35 -07:00
Devin AI
adf5637fdf chore: merge litellm_internal_staging into litellm_techdebt_20260829
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 11:31:44 +00:00
Mateo Wang
6a3333d3c8
Merge pull request #38747 from BerriAI/litellm_aws_partition_helper
fix(aws): build every AWS endpoint and ARN from the region's partition (aws-cn, aws-us-gov)
2026-08-29 04:10:59 -07:00
Mateo Wang
39e4f1ae13
Merge pull request #38727 from BerriAI/litellm_aws_external_id_embed_sagemaker
fix(aws): forward aws_external_id in Bedrock embeddings and SageMaker credential loading
2026-08-29 03:30:32 -07:00
Mateo Wang
1fbf34e559
Merge pull request #38744 from BerriAI/litellm_fix_bedrock_batch_zero_count_retire
fix(bedrock): map real batch record counts and guard zero-count retire
2026-08-29 03:12:49 -07:00
mateo-berri
850b18d214 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_tag_routing_reads_merged_metadata_tags
# Conflicts:
#	tests/openai_endpoints_tests/test_e2e_openai_responses_api.py
2026-08-29 02:34:51 -07:00
mateo-berri
1947c65081 test(aws): type the new partition test parameters 2026-08-29 02:27:18 -07:00
mateo-berri
d5b5aca498 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_bedrock_batch_zero_count_retire 2026-08-29 02:21:28 -07:00
mateo-berri
2d11870793 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_batch_id_fallback_pin
# Conflicts:
#	tests/openai_endpoints_tests/test_e2e_openai_responses_api.py
2026-08-29 02:19:45 -07:00
mateo-berri
cd5cf84aac Merge branch 'litellm_internal_staging' into litellm_aws_partition_helper 2026-08-29 02:18:16 -07:00
mateo-berri
b0d485568b test(batches): pin the poller against retiring zero-count completed batches 2026-08-29 02:00:11 -07:00
mateo-berri
66947fcf7c test(responses): expect the bad-temperature 400 on a non-reasoning model 2026-08-29 01:58:15 -07:00
mateo-berri
8cf090b368 fix(router): arm the provider-scoped fallback pin only on resource-operating handlers 2026-08-29 01:46:33 -07:00
mateo-berri
7fbcd9c3ed fix(bedrock): treat partial record counts as unknown on batch retrieve 2026-08-29 01:40:07 -07:00
mateo-berri
81eb014e71 test(responses): use a reasoning-legal temperature in the gpt-5.5 extra_body merge test 2026-08-29 01:36:45 -07:00
mateo-berri
2968c246b9 test(responses): repair two tests broken by the gpt-5 temperature gate
PR #38593 stopped forwarding temperature to reasoning models, which left
test_extra_body_merges_with_request_data raising UnsupportedParamsError
and test_bad_request_bad_param_error no longer getting a rejection from
OpenAI because drop_params now eats the param. Both repairs are the same
hunks PR #38739 carries, so the branches merge clean in either order
2026-08-29 01:32:09 -07:00
mateo-berri
9da766a1f9 test: use a non-reasoning model for the responses bad-param e2e fixture 2026-08-29 01:22:49 -07:00
mateo-berri
ad8c1457d1 fix(aws): build every AWS endpoint and ARN from the region partition
Adds litellm/litellm_core_utils/aws_partition.py mapping a region to its
AWS partition (aws, aws-cn, aws-us-gov, and the iso partitions), its DNS
suffix, and its ARN prefix, and uses it at every AWS host and ARN build
site: bedrock (runtime, agent, agentcore, legacy client, batches, files,
realtime), sagemaker, polly, secrets manager, s3 log uploads, bedrock
passthrough routes, and rag ingestion. ARN detection now accepts
arn:aws-cn: and arn:aws-us-gov: prefixes.

STS region resolution now falls back to the configured aws_region_name
after the aws_sts_endpoint host and the AWS_REGION/AWS_DEFAULT_REGION env
vars, so cn and gov role assumption no longer silently signs against
us-west-2.

A partition sweep test walks every endpoint builder with cn regions and
asserts no amazonaws.com host or arn:aws: prefix comes out, plus an AST
guard that fails on any new f-string hardcoding either literal.
2026-08-29 01:21:59 -07:00
mateo-berri
6301a520a6 test: pin reasoning effort none in the responses extra_body fixture 2026-08-29 01:07:24 -07:00
mateo-berri
5d34fb20ff fix(bedrock): map real batch record counts and guard zero-count retire 2026-08-29 01:05:36 -07:00