kerry
0a47fe160d
Merge remote-tracking branch 'origin/main' into litellm_mantle_gpt5_verbosity
2026-09-17 00:06:50 +00:00
Mateo Wang
09a188b583
Merge pull request #41094 from BerriAI/litellm_model_group_info_proxy_admin_all_models
...
fix(proxy): show all model groups to proxy admins in /model_group/info
2026-09-16 17:06:44 -07:00
Mateo Wang
913ef6ed49
Merge pull request #33856 from BerriAI/litellm_azure_ai_responses_native
...
fix(azure_ai): route Responses API to native /openai/v1/responses for Foundry Models
2026-09-16 17:06:29 -07:00
kerry-berri
f93b31679b
Merge pull request #41503 from BerriAI/litellm_fix_interrupted_anthropic_reasoning_usage
...
fix(streaming): estimate interrupted Anthropic stream usage from reasoning_content
2026-09-16 17:04:59 -07:00
Mateo Wang
6cdf398bea
Merge pull request #41504 from BerriAI/litellm_bedrock_agent_runtime_strip_virtual_key
...
fix(proxy): stop forwarding LiteLLM credential headers on Bedrock agent-runtime passthrough
2026-09-16 17:02:17 -07:00
Mateo Wang
319f427c40
Merge pull request #41201 from BerriAI/litellm_gemini_37_38_flash_no_minimal_thinking
...
fix(gemini): map minimal thinking to low for Gemini 3.7 and 3.8 Flash
2026-09-16 16:52:08 -07:00
mateo-berri
ada0a1ad3a
fix(azure_ai): strip the azure_ai/ prefix when a Responses call is remapped to azure
...
A catalog OpenAI name on an .openai.azure.com host (or with AZURE_AI_API_BASE set to one) is remapped from azure_ai to azure before the Responses request is built, and the azure_ai/ prefix stayed in the wire model, so Azure answered DeploymentNotFound. The Azure Responses config now strips azure_ai/ next to responses/ and o_series/.
2026-09-16 16:49:29 -07:00
yucheng-berri
c5325b1492
Merge pull request #40596 from BerriAI/litellm_lit_7470_rate_limit_fallback_pristine_data
...
fix(proxy): retry rate-limit fallbacks from a pristine request snapshot
2026-09-16 16:47:35 -07:00
yucheng-berri
672f43fd54
Merge pull request #41356 from BerriAI/litellm_lit7836_call_id_endpoint_logs
...
fix(proxy): carry litellm_call_id through endpoint specific error logs and failure responses
2026-09-16 16:43:12 -07:00
mateo-berri
41737aeda8
test(proxy): drop the docstring from the agent-runtime passthrough regression class
2026-09-16 16:37:32 -07:00
kerry
e8f246bb6b
fix(responses_bridge): forward verbosity as text.verbosity
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:32:32 +00:00
Mateo Wang
390ab45448
Merge pull request #37506 from BerriAI/litellm_dashscope_reasoning_effort
...
LiteLLM Rust / rust-wheel (push) Waiting to run
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests / proxy-server (push) Waiting to run
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests / caching-local (push) Waiting to run
Unit Tests / core-utils (push) Waiting to run
Unit Tests / enterprise-package (push) Waiting to run
Unit Tests / enterprise-routing (push) Waiting to run
Unit Tests / integrations (push) Waiting to run
Unit Tests / All Other Providers (push) Waiting to run
Unit Tests / Vertex AI (push) Waiting to run
Unit Tests / misc (push) Waiting to run
Unit Tests / proxy-auth (push) Waiting to run
Unit Tests / proxy-endpoints (push) Waiting to run
Unit Tests / proxy-extras (push) Waiting to run
Unit Tests / proxy-infra (push) Waiting to run
Unit Tests / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
fix(dashscope): forward reasoning_effort to the provider
2026-09-16 16:29:25 -07:00
kerry
884e96958c
fix(streaming): cap estimated reasoning tokens to the provider total and cover dict chunks
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:27:48 +00:00
kerry
15c18ad6cd
fix(bedrock_mantle): accept verbosity on gpt-5.x chat completions
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:27:07 +00:00
joshua-berri
41410e9556
Merge pull request #41364 from BerriAI/litellm_fix_mcp_auth_fail_closed_4501
...
fix(mcp): fail closed on missing upstream credentials
2026-09-16 23:26:38 +00:00
mateo-berri
3712d8de92
Merge remote-tracking branch 'origin/main' into litellm_model_group_info_proxy_admin_all_models
LiteLLM Rust / rust-wheel (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
2026-09-16 16:18:45 -07:00
mateo-berri
2a9fa48730
fix: expand wildcard deployments for proxy admins on /model_group/info
2026-09-16 16:18:44 -07:00
mateo-berri
a1ad95dbbd
fix(gemini): read the minimal thinking floor from the cost map and cover the /v1/messages bridge
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
2026-09-16 16:13:41 -07:00
kerry
43514a7ffe
Merge remote-tracking branch 'origin/main' into litellm_fix_interrupted_anthropic_reasoning_usage
2026-09-16 23:13:06 +00:00
kerry
02bccfd89f
fix(streaming): fill text_tokens when reasoning is counted from stream content
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:13:05 +00:00
yuneng-jiang
765e6e498d
Merge pull request #41402 from BerriAI/litellm_/buildkite-litellm-e2e-setup-ff714d
...
feat(e2e): make the provider cache reusable across builds and mount Bedrock behind it
2026-09-16 16:12:29 -07:00
yuneng-jiang
030901211e
Merge pull request #41360 from BerriAI/litellm_unpin_together_ai_serverless_model
...
test(together_ai): move request-shape checks to the mapped file, drop the live ones
2026-09-16 16:07:33 -07:00
yuneng-jiang
fddf83a2ac
Merge pull request #41487 from BerriAI/litellm_isolate_generic_api_ndjson_test
...
test(logging): pick this test's own records out of the shared log batch
2026-09-16 16:06:47 -07:00
ryan-crabbe-berri
be6e7ef173
Merge pull request #41345 from BerriAI/litellm_remove_duplicate_user_budget_hook
...
fix(proxy): remove duplicate user budget hook that 429'd zero-cost models
2026-09-16 16:06:41 -07:00
yucheng-berri
659c4ce5a1
Merge pull request #40669 from BerriAI/litellm_otel_passthrough_trace_propagation
...
fix(otel): propagate W3C trace context on HTTP and WebSocket passthrough
2026-09-16 16:02:46 -07:00
mateo-berri
02d9aae8c8
fix(proxy): stop forwarding LiteLLM credential headers on Bedrock agent-runtime passthrough
...
The agent-runtime branch of /bedrock/{endpoint} (agents, knowledgebases, flows,
retrieveAndGenerate, rerank, generateQuery, optimize-prompt) forwarded every
caller header to AWS next to the SigV4 signature, so a LiteLLM key presented in
x-api-key or x-litellm-api-key reached bedrock-agent-runtime verbatim. Build the
upstream header set explicitly: drop LiteLLM credential headers by name and any
authenticated secret by value, keep the remaining caller headers, and let the
signed headers win on collisions.
2026-09-16 16:01:52 -07:00
Yuneng Jiang
afb28540bb
fix(e2e): keep the CLI determinism test out of the in-cluster suite
...
It drives the real CLI for several seconds. The edge stamps every
upstream call with PYTEST_CURRENT_TEST, a process-global that names
whichever test the worker is in when the call arrives rather than the one
that made it, so a test that holds a worker that long collects other
tests' in-flight calls. Build 234's key report credits this test with 20
Bedrock and 7 Anthropic misses, and it makes no provider call at all.
Those misattributed calls take the wrong test id into the cache key and
write recordings under it, so the test was polluting the shared corpus it
exists to protect.
Deselected unless E2E_CLI_DETERMINISM is set, the same opt-in shape the
managed-files, prompt-caching and redis-chaos markers already use. The
attribution bug itself is older than this branch and is reported, not
fixed here.
2026-09-16 16:01:32 -07:00
mateo-berri
8045794c37
Merge remote-tracking branch 'origin/main' into litellm_dashscope_reasoning_effort
2026-09-16 15:59:08 -07:00
yuneng-jiang
ceaa0adef8
Merge branch 'main' into litellm_unpin_together_ai_serverless_model
2026-09-16 15:57:17 -07:00
yuneng-jiang
7f83650a41
Merge branch 'main' into litellm_isolate_generic_api_ndjson_test
2026-09-16 15:56:53 -07:00
kerry
4b70696afa
fix(streaming): estimate interrupted Anthropic stream usage from reasoning_content
...
Interrupted Anthropic streams that die before message_delta were billed at
the message_start placeholder (any value above 1 was trusted) or at 0 when
the partial response was reasoning-only, because the token_counter fallback
only looked at visible text. Reset the placeholder whenever no finish_reason
or second usage event arrived, fold the already-counted reasoning tokens into
the fallback estimate, and drop the stale completion_tokens_details so cost
is computed from the recovered count
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 22:49:25 +00:00
ryan
dae16264c1
test(auth): cover over-budget user on zero-cost vs paid model in common_checks
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 15:42:15 -07:00
ryan
287bbaa6c1
fix(proxy): remove duplicate user budget hook that 429'd zero-cost models
...
_PROXY_MaxBudgetLimiter re-checked spend:user:{id} against user_max_budget in
async_pre_call_hook without the zero-cost model exemption that
_user_max_budget_check applies in auth, so free models were rejected with
"Max budget limit reached." once a user was over budget. Auth already owns
this check, so the hook is deleted rather than taught the exemption again
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 15:42:15 -07:00
Yuneng Jiang
99545b5f26
Merge remote-tracking branch 'origin/main' into litellm_/buildkite-litellm-e2e-setup-ff714d
2026-09-16 15:15:22 -07:00
Yassin Kortam
a6e87526f8
Merge pull request #40843 from elifozdamar/litellm_fix_nonstream_parallel_slot_release
...
fix(proxy): release completed max-parallel slots promptly
2026-09-16 15:13:59 -07:00
mateo-berri
ea5887d4bb
test: cover proxy_admin_viewer sessions in the model_group/info admin regression test
2026-09-16 15:10:48 -07:00
Yuneng Jiang
76c0f8db1d
chore(e2e): report the key components behind a mount that never converges
...
Builds 232 and 233 held the Bedrock hit rate at 9% with the Claude Code
driver already sending byte-identical requests and headers, so something
between the proxy's ingress and the upstream still moves per build and
the flat key cannot say what.
Emit a digest per key component next to the counters: the test id, the
method, the URL, each keyed header, the whole body, and one digest per
top-level JSON body field. Values are digested, so no payload or
credential reaches the artifact. Diffing two builds' artifacts names the
field that moved.
Diagnostic, to be removed once it has answered.
2026-09-16 15:01:08 -07:00
Yassin Kortam
f62f140f75
Merge pull request #41403 from BerriAI/litellm_key_total_spend
...
feat(proxy): expose lifetime total_spend on virtual keys
2026-09-16 14:49:02 -07:00
Yassin Kortam
2bfa10b6d9
Merge pull request #41311 from BerriAI/litellm_key_status_filter_deleted_lookup
...
feat(keys): filter /key/list by active, expired, revoked or deleted status and serve deleted keys from /key/info
2026-09-16 14:48:34 -07:00
Yassin Kortam
95abc9fb0b
Merge pull request #41330 from BerriAI/litellm_team_model_max_budget_v2
...
feat(team): team-level model_max_budget with key-level overrides
2026-09-16 14:48:29 -07:00
Yassin Kortam
314c0d71a5
Merge pull request #41327 from BerriAI/litellm_s3_log_prompts_only
...
feat(s3): add s3_log_prompts_only option to log prompts without responses
2026-09-16 14:48:18 -07:00
yujonglee
cc99ad00dd
Merge pull request #41489 from BerriAI/litellm_ocr_fileio_core
...
refactor(ocr): move file preparation from the python bridge into litellm-core
2026-09-16 14:45:09 -07:00
mateo-berri
eaeeb3a9bc
Merge remote-tracking branch 'origin/main' into litellm_dashscope_reasoning_effort
2026-09-16 14:41:08 -07:00
mateo-berri
da844e3dd2
Merge remote-tracking branch 'origin/main' into litellm_model_group_info_proxy_admin_all_models
2026-09-16 14:40:10 -07:00
ryan-crabbe-berri
4439631e05
Merge pull request #41379 from runjivu/fix/fallback-budget-check
...
fix!: re-check budget on router fallback targets
2026-09-16 14:39:56 -07:00
Yassin Kortam
cd08c65002
Merge pull request #41425 from BerriAI/litellm_streaming_buffer_release_on_scan
...
feat(guardrails): release buffered stream chunks after each passing scan
2026-09-16 14:37:56 -07:00
Yassin Kortam
16bbff6643
Merge pull request #41474 from BerriAI/litellm_router_tpm_rpm_count_before_headers_v2
...
fix(router): count TPM/RPM usage before building rate-limit headers
2026-09-16 14:37:04 -07:00
Yuneng Jiang
0de187e76c
style(test): annotate the new locals as Final
2026-09-16 14:34:13 -07:00
Yuneng Jiang
515bf8c9d5
refactor(test): validate cost-map entries into a typed model
...
The selector read raw cost-map dicts as `Mapping[str, Any]`. It now validates
each together_ai entry into a frozen Pydantic model and takes the two
capabilities as keyword booleans, so nothing in the helper is coarsely typed
or stringly addressed.
2026-09-16 14:32:08 -07:00
Mateo Wang
6edb549dbd
Merge pull request #41112 from BerriAI/litellm_registry_audit_2026_09_14
...
fix(models): rolling registry audit: Gemini latest aliases, Nova cache pricing, OpenRouter/Together sync, Mistral GLM 5.3, Azure snapshots, Grok caching
2026-09-16 14:31:50 -07:00
