Commit graph

50724 commits

Author SHA1 Message Date
yucheng
87263cefca Merge remote-tracking branch 'origin/main' into litellm_post_call_guardrail_context
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/llms/openai/responses/test_openai_responses_guardrail_handler.py
2026-09-17 05:08:02 +00:00
Yuneng Jiang
a0869fe835
test(budgets): avoid mutable fixture state 2026-09-16 22:06:31 -07:00
yucheng
719d7a1983 test(proxy): cover inherited moderation overrides through during_call_hook
Replaces the capability flag assertion with a behavioral test that dispatches an
async_moderation_hook inherited from a parent class, and drops the dispatch
docstring that restated the code

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 05:00:01 +00:00
Yuneng Jiang
5c41e0b8dc
test(budgets): cover management null handling 2026-09-16 21:42:55 -07:00
yuneng-jiang
5ef40a630b
Merge pull request #41527 from BerriAI/litellm_/monitor-cci-failures-ac97a9
test: fix seven tests left stale by #41311, #41337, #39996, #41310, #41289 and #41315
2026-09-16 21:25:51 -07:00
yassin
45ceb56110 fix(router): validate max_parallel_requests_queue_size as a non-negative integer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 04:05:41 +00:00
yucheng-berri
821bcf5d78
Merge pull request #41140 from BerriAI/litellm_otel_v2_langfuse_user_session_tags 2026-09-16 20:55:42 -07:00
Yuneng Jiang
c6023b4eec
test: pin the post-#41289 cooldown contract and scroll the auto-router select spec
test_router_fallbacks_with_cooldowns_and_dynamic_credentials expected a
caller-supplied credential to register its own deployment and cool it down.
#41289 stopped registering it, so cooldown logic skips that id and the
assertion can never hold. The test now asserts what the router guarantees
today: a 429 to a forwarded credential cools down none of the shared
deployments, the next credential is still served, and a 429 owned by a shared
deployment still cools it down. The final live OpenAI call becomes a mock

The auto-router template spec assumed the Add Auto Router form left room
below the Template select at 1280x900. #41315 added classifier fields above
it, so the options opened upward. The spec now scrolls the trigger to the top
of the dialog and asserts it sits in the upper half before checking placement
2026-09-16 20:52:19 -07:00
Yujong Lee
85e70ea374 fix(ocr): await blocking preparation on cancellation 2026-09-16 20:49:31 -07:00
yuneng-jiang
54fa790e20
Merge pull request #41551 from BerriAI/litellm_cadence_319f427_key_lifecycle_delete
test(e2e): read a deleted key back as deleted, not as a 404
2026-09-16 20:48:55 -07:00
yuneng-jiang
3c34b92594
Merge pull request #41524 from BerriAI/litellm_aws_rotation_values
test(aws): verify rotated secret value
2026-09-16 20:41:12 -07:00
yucheng-berri
375cd4a668
Merge pull request #41498 from BerriAI/litellm_otel_indexed_messages_span_headroom 2026-09-16 20:40:24 -07:00
ryan-crabbe-berri
8b64f1ef03
Merge pull request #41525 from BerriAI/litellm_team_admin_rpm_budget_fields
feat(proxy): let team admins edit rpm_limit and max_budget when enabled
2026-09-16 20:39:05 -07:00
yucheng
2cbfd280e9 Merge remote-tracking branch 'origin/main' into litellm_prompt_injection_async_llm_check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	litellm/proxy/proxy_server.py
2026-09-17 03:32:34 +00:00
tin-berri
d18e06f736
Merge pull request #41508 from BerriAI/litellm_1789600151_discover_context_limits
feat(router): discover token limits for hosted OpenAI-compatible models
2026-09-16 20:29:57 -07:00
Yujong Lee
e0ce998091 fmt 2026-09-16 20:21:51 -07:00
yassin
6be9c4a978 refactor(router): compose DeploymentSemaphore over asyncio.Semaphore instead of subclassing it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 03:17:49 +00:00
yassin
fe0eee6451 fix(anthropic): keep prompt cache prediction supported for queue-bounded deployments
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 03:02:42 +00:00
yassin
446fadc4c7 feat(router): bound the max_parallel_requests wait queue and return 429 on overflow
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 02:50:45 +00:00
Yujong Lee
edfa01da81 refactor(ocr): mirror Python provider layout and preserve tests 2026-09-16 19:48:26 -07:00
yuneng-berri
44a0e16c81
test(e2e): read a deleted key back as deleted, not as a 404
/key/info now serves a deleted key from the archive with status deleted
instead of answering 404, so the delete test's convergence predicate never
settled and the read timed out against a 200 it kept discarding.

The predicate now waits for status deleted through the same
_key_info_everywhere helper the rest of the file uses, and KeyInfo carries
the status field. The chat-rejection assertion after it is unchanged, so
the test still proves the key stops serving.
2026-09-17 02:38:27 +00:00
yucheng
3c000e4ffb fix(proxy): run prompt injection heuristics on a dedicated bounded executor
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 02:38:13 +00:00
yucheng
d50bac391e test(proxy): cover startup router wiring for registered prompt injection detectors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 02:26:12 +00:00
yassin
8691a1e190 fix(vault): cache the secret body per url so mutations evict every field
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 02:23:50 +00:00
yassin
4694bd0c63 fix(vault): key the secret cache by url and data field
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 02:08:58 +00:00
Yujong Lee
03cd00fbb1 refactor(rust): standardize messages errors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 02:08:26 +00:00
yucheng
fc77914df3 test(proxy): type the moderation override stub in hook detection tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 01:59:04 +00:00
yassin
bb9ff8cb2c fix(bedrock): keep realtime SDK error range inside websocket close reason
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 01:57:33 +00:00
kerry
6a9ae2bba2 ci(aws-partition): count allowlisted literal occurrences so duplicates in allowed files fail
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 01:47:33 +00:00
yassin
15bfe8f28a feat(vault): add separate login and secret namespaces for HashiCorp Vault
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 01:41:48 +00:00
yassin
0259e8c7d5 fix(bedrock): support aws-sdk-bedrock-runtime 0.10 and 0.11 in the realtime handler
The bedrock-realtime extra pinned aws-sdk-bedrock-runtime 0.7.x, whose Config and BedrockRuntimeClient surface is gone in 0.11. The handler now resolves AsyncBedrockRuntimeConfig, builds AsyncBedrockRuntimeClient with the awscrt duplex transport, closes the client when the session ends, and tells an absent SDK apart from an installed but unsupported version. Moves the pin to >=0.10.0,<0.12.0 with the awscrt extra

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 01:40:52 +00:00
yucheng
b1255a6f2c fix(proxy): run prompt injection heuristics off the event loop and dispatch llm_api_check moderation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 01:33:54 +00:00
kerry
302394edff ci: gate hardcoded commercial AWS partition literals and test us-gov endpoint builders
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 01:33:43 +00:00
Yujong Lee
4e5a9efd9d feat(rust): map anthropic messages transforms 2026-09-16 18:25:40 -07:00
mateo-berri
47b2479c94 fix(bedrock): gate Invoke tool search on the model map's supports_tool_search flag
The Bedrock InvokeModel transformations decided whether to send the
tool-search-tool-2025-10-19 beta from hardcoded model name lists (a pattern
list on the messages path, an "opus-4" substring on the chat path), so Opus 4.8,
Opus 5 and Sonnet 5 never got the beta on the messages path, Opus 5 and Sonnet 5
never got it on the chat path, Opus 4.1 got it without support, and
/v1/model/info reported supports_tool_search as unset for all three.

Both paths now read the model map through one shared helper: the Bedrock
entries for Opus 4.8, Opus 5 and Sonnet 5 carry supports_tool_search
explicitly, and a claude-tool-search fallback rule flags Claude 4.5 and newer
for unmapped ids, inference-profile ARNs and mapped entries with no opinion,
so the next Claude gets the beta with no code change. An explicit false on a
resolved entry still wins.
2026-09-16 18:23:14 -07:00
Yassin Kortam
351a54e849
Merge pull request #41507 from BerriAI/litellm_attribute_router_rejected_spend_provider
fix(spend_tracking): attribute router-rejected requests to the model group provider
2026-09-16 18:21:44 -07:00
ryan-crabbe-berri
fc13cea479 fix(proxy): refuse a team admin's budget write when the budget changed mid-request
The keep-or-lower check compares against the budget update_team read, so the write now only lands while the stored max_budget still matches it and answers 409 otherwise. A concurrent proxy admin cut can no longer be overwritten with a higher value.
2026-09-16 18:11:53 -07:00
Yujong Lee
2414d1f028 feat(rust): scaffold anthropic stream transformation 2026-09-16 18:10:28 -07:00
yassin
65d0f3a03d fix(terraform): mirror /transcribe into the AWS and GCP gateway prefix lists
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 01:07:57 +00:00
yassin
8533dc9673 fix(helm): route /transcribe to the gateway and drop pinned botocore operation from test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 01:06:15 +00:00
yucheng
7815719de7 fix(guardrails): stream Prompt Security post_call redactions in incremental_diff mode
Forward streaming_transform_mode from guardrail litellm_params into PromptSecurityGuardrail so incremental_diff is reachable from config; the default stays block_only. In incremental_diff the guardrail now returns stream_holdback_chars alongside the rewritten texts so that a value split across streamed chunks (or across an abbreviation period) is never partially released before the vendor rewrite arrives. Each response text gets its own protect call so modified_text maps back to the right choice when n > 1, and custom_guardrail no longer logs a clean response as mask just because the guardrail attached holdback metadata

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 01:06:12 +00:00
Mateo Wang
2445bdd2b5
Merge pull request #40934 from BerriAI/litellm_fix_ocr_native_multipage_pdf
fix(logging): scan each log record once and collapse base64 payloads before the secret regex
2026-09-16 18:06:11 -07:00
yassin
e06c81665f refactor(spend_tracking): type the get_logging_payload parameters
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 01:02:03 +00:00
mateo-berri
c2f77fd358 refactor(a2a): resolve the relay's Entra hop bearer inside the a2a provider helper 2026-09-16 17:56:42 -07:00
yuneng-jiang
38676aa599
Merge pull request #41078 from BerriAI/litellm_integration_extensions
test: add extension and browser integration contracts
2026-09-16 17:55:25 -07:00
yuneng-jiang
e927b63211
Merge pull request #41520 from BerriAI/litellm_/attribution-investigation-a81211
fix(e2e): bind provider-cache recordings to the deployment's test, not the serving process
2026-09-16 17:52:03 -07:00
ryan-crabbe-berri
d3f0607820 test(ui): find the max_budget checkbox by its new label 2026-09-16 17:50:57 -07:00
yucheng-berri
0add8c0083
Merge pull request #41495 from BerriAI/litellm_converted_stream_post_call_hook
fix(utils): run post-call deployment hook on converted chat streams
2026-09-16 17:49:14 -07:00
yucheng
79aae7f062 refactor(otel v2): build Langfuse trace attributes from pairs to satisfy the type discipline gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 00:49:12 +00:00
Yuneng Jiang
02ced74540
test: fix five tests left stale by #41311, #41337, #39996 and #41310
Every one of these fails on main's own scheduled CircleCI run with the same
assertion as on any PR, and each traces to a merged behavior change that
never updated the test that pinned the old behavior

- tests/integration/_support/client.py: #41311 made /key/info serve deleted
  keys from the archive with status deleted, so the scenario teardown asserts
  the live row is gone and the readback reports deleted instead of a 404.
  This alone accounts for nine integration-management and one
  integration-providers failure
- tests/integration/authorization/test_warmed_policy.py: #39996 made team
  admins unable to edit any team field unless a proxy admin allow-lists it,
  and tpm_limit is the only field it accepts today. The demotion test now
  enables tpm_limit for the scenario and edits that instead of team_alias
- tests/llm_responses_api_testing/test_base_responses_api_streaming_iterator.py:
  #41337 reads usage off the terminal response and copies the event when it
  is missing, which a Mock(spec=ResponsesAPIResponse) cannot survive. The
  four mocks now carry a usage object
- tests/test_openai_endpoints.py: #41310 lengthened the access-denied
  message, and the test matched against the ExceptionInfo repr, which
  saferepr truncates in the middle. It now matches the exception text
- tests/local_testing/test_text_completion.py: Together no longer serves
  Qwen2-1.5B serverless, the cheapest cost-map row. The test mocks the
  completions call and asserts the request litellm builds, so a vendor
  catalog rotation cannot fail it again

test_router_fallbacks_with_cooldowns_and_dynamic_credentials is deliberately
untouched: it passes and fails on main with identical code, and the failing
path is a product question about whether dynamic-credential 429s cool down
2026-09-16 17:47:27 -07:00