Commit graph

50986 commits

Author SHA1 Message Date
Yuneng Jiang
d67c7894dd
fix(ui): keep recorded order when an untimed guardrail shares a phase 2026-09-15 22:25:30 -07:00
Yuneng Jiang
8bb496154a
test(ui): scope lifecycle assertions with within instead of parentElement
The four .parentElement reads in the new lifecycle tests pushed
testing-library/no-node-access to 712 against a 707 budget, failing
frontend-lint. The rows now carry data-testid="lifecycle-row" and the test
picks a row with within(), which keeps the assertion tied to the specific row
rather than the whole panel and takes the count back to 707.
2026-09-15 22:18:42 -07:00
Joshua Valluru
e035682ed1 refactor(auth): separate JWT identity and OAuth authorization 2026-09-15 22:15:57 -07:00
Yuneng Jiang
5bd27d1d99
Merge remote-tracking branch 'origin/main' into litellm_fix_integration_conftest_import 2026-09-15 21:45:12 -07:00
Yuneng Jiang
d20469b0dd
Merge remote-tracking branch 'origin/main' into litellm_fix_guardrail_lifecycle_untimed_entries 2026-09-15 21:44:22 -07:00
Yuneng Jiang
51a243e3ce
fix(ui): keep untimed guardrail entries on the request lifecycle
#39050 changed RequestLifecycle from sorting every entry with
(a.start_time ?? 0) to filtering on isTimed, which drops any entry whose
start_time/end_time are null. That was the right call for the not_run entries
the PR introduced, but it also drops entries that DID run and simply carry no
timing, and those are pre-existing: add_standard_logging_guardrail_information_to_request_data
defaults start_time, end_time and duration to None, and the conduct guardrail
passes none of them. One such entry used to draw the whole four-row lifecycle
and now draws nothing, so an admin opening that log sees an empty
Request Lifecycle panel.

An entry now stays on the lifecycle when it is timed OR when it ran, so not_run
keeps the exclusion #39050 wanted and every other shape comes back. Offsets are
number | null and render as an em dash rather than a fabricated T+0ms, which is
what a null minus a null used to produce on the base. Entries without timing
sort after the timed ones and the base time comes from the timed entries, so
real offsets are unchanged.

The two new tests fail on the base component and pass here; #39050's own
not_run tests keep passing untouched, which is what makes this additive rather
than a revert.
2026-09-15 21:44:19 -07:00
yassin
252c69b532 fix(router): snapshot shadow kwargs per target so concurrent shadows never share metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 04:31:43 +00:00
tin-berri
a8979fe054
Merge pull request #41371 from BerriAI/litellm_forecast_simplify_advanced_ui
fix(ui): simplify Capability and Fuse advanced routing options
2026-09-15 21:19:50 -07:00
kerry-berri
5960881640
Merge pull request #41337 from BerriAI/litellm_fix_responses_stream_absent_usage_recount
fix(responses): recount tokens when a streamed response completes without usage
2026-09-15 21:19:42 -07:00
yuneng-jiang
bbffddd517
Merge pull request #41366 from BerriAI/litellm_/back-002-litellm-e2e-replay-8de5b1
fix(e2e): record cookie-setting provider responses and keep prompt-caching tests live
2026-09-15 21:18:14 -07:00
Yuneng Jiang
3b0fbc426d
fix(tests): resolve the integration support package without run.py's PYTHONPATH
tests/integration/conftest.py imported the bare `integration` package. Because
tests/__init__.py and tests/integration/__init__.py both exist, pytest's default
prepend import mode puts only the repo root on sys.path, so that name resolved
only under the PYTHONPATH that tests/integration/run.py injects. Every other
invocation died at conftest import with
ModuleNotFoundError: No module named 'integration' and exit 4, including the
command test_oci_integration.py documents in its own docstring.

The imports now use the tests.integration._support path that pytest actually
resolves, matching the 120 other `from tests.` imports in the suite. run.py's
PYTHONPATH still works because it already puts the repo root on the path.

tests/code_coverage_tests/test_integration_suite_imports.py collects every file
under tests/integration with PYTHONPATH scrubbed and asserts a non-zero
collection count, so an unresolvable import fails the code-quality job instead
of only the developers who run these files by hand. CI runs the three
pre-existing files through the allowlist rather than executing them, which is
why nothing caught this.
2026-09-15 21:10:08 -07:00
yuneng-jiang
e3c1f78e28
Merge branch 'main' into litellm_/back-002-litellm-e2e-replay-8de5b1 2026-09-15 21:08:29 -07:00
yassin
0f7ed4433b fix(router): snapshot shadow kwargs before fan-out so shadows never see primary mutations
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 04:04:01 +00:00
Tin Chi Lo
9d640b86ed fix(ui): simplify Capability and Fuse routing options 2026-09-15 20:58:38 -07:00
kerry
ff878e7df0 refactor(responses): copy the terminal event instead of mutating stubbed chunks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 03:54:04 +00:00
yassin
9cabde90dd test(router): cover _run_silent_experiment directly for router coverage gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 03:48:32 +00:00
kerry
7cc07d437a fix(responses): build the billed terminal response immutably and guard the cache dump
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 03:47:03 +00:00
berriai-litellm-provider-info-sync[bot]
641dcd5f8f
chore(prices): sync Google Gemini prices: 1 model
gemini/gemini-robotics-er-2-preview: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost, input_cost_per_token_batches, output_cost_per_token_batches
2026-09-16 03:46:34 +00:00
Joshua Valluru
258176de76 fix(mcp): validate rendered static credential payloads 2026-09-15 20:32:25 -07:00
yuneng-jiang
174c1ac4ed
Merge pull request #41348 from BerriAI/litellm_fix_models_reload_test
fix(e2e): expect models filters to persist after reload
2026-09-15 20:31:47 -07:00
yassin
06fcc1f733 feat(router): stream shadow traffic and fan out silent_model to multiple targets
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 03:28:08 +00:00
yuneng-jiang
9d54c2e159
Merge pull request #41359 from BerriAI/litellm_fix_budget_reset_decrement_tests
test(proxy): assert budget resets decrement the cleared spend
2026-09-15 20:27:35 -07:00
yuneng-jiang
ef06790e39
Merge pull request #41358 from BerriAI/litellm_fix_stale_router_tests
test(router): ignore deployment-selection logs in the fallback log assertion
2026-09-15 20:27:21 -07:00
Yuneng Jiang
a6fb21c3f8
fix(e2e): record cookie-setting provider responses and keep prompt-caching tests live
The first cache-enabled litellm-e2e build (211) showed three gaps in the shared provider cache:

Every OpenAI response carries Cloudflare bot-management Set-Cookie headers, and the capture rejected any response with Set-Cookie, so no OpenAI response was ever recorded (179 of 372 misses rejected). The edge already withholds Set-Cookie from the proxy, so drop it before validating and storing instead of rejecting.

The provider prompt-caching tests need fresh provider state: a replayed priming response reports cache creation rather than a cache read, and the TPM test then trips the key limit. Mark both modules provider_live.

TestApiBaseSeam::test_live_mode_returns_none ran inside the cache-enabled runner and saw the shared edge; isolate it from E2E_PROVIDER_CACHE.
2026-09-15 20:26:20 -07:00
kerry-berri
4b84fa9230
Merge pull request #41336 from BerriAI/litellm_fix_anthropic_stream_absent_usage
fix(anthropic): tolerate message_delta events without usage when streaming
2026-09-15 20:23:01 -07:00
Yuneng Jiang
8c046e13bd
test(together_ai): move request-shape checks to the mapped file, drop the live ones
Together moved openai/gpt-oss-20b off serverless and three tests in
test_completion.py died on a live 400. None of them needed Together to be up:
streaming is already covered live by tests/e2e/llm_translation/test_together_ai_e2e.py,
which picks its model from the cost map instead of pinning one, and the other
two are request-shape questions. Delete all three and assert the two shapes in
the mapped transformation file: the provider prefix is stripped without eating
the rest of a slashed model name, and custom role wrappers never reach the
request.
2026-09-15 20:17:42 -07:00
tin-berri
3b14631f06
Merge pull request #41315 from BerriAI/litellm_capability_fuse_classifier_ui
feat(ui): configure capability and Fuse v2 classifiers
2026-09-15 20:13:57 -07:00
kerry
7121e64db4 fix(responses): type the dict terminal response so the estimated usage is billed
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 03:13:53 +00:00
yucheng
581c613f66 fix(proxy): keep litellm_call_id on shaped errors and list_batches failure hook
Already shaped ProxyException and HTTPException errors passing through the moderations, audio speech, Anthropic Messages, and handle_exception_on_proxy paths now answer with the x-litellm-call-id header the route logged under, without overwriting a header the exception was raised with. The GET /v1/batches failure hook receives the resolved request data so the spend log request_id matches the response header and the error log

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 03:02:50 +00:00
Yassin Kortam
d108cdc431
Merge pull request #38254 from BerriAI/litellm_fix_xai_responses_instructions 2026-09-15 19:55:52 -07:00
Joshua Valluru
be506936bd fix(mcp): preserve explicit caller authorization credentials 2026-09-15 19:52:40 -07:00
Yuneng Jiang
1aa2e19ee4
test(together_ai): stop depending on a serverless model we do not control
Together moved openai/gpt-oss-20b off serverless, so three tests started
failing with a 400 model_not_available from the live API. None of them was
really testing Together: they cover provider-prefix parsing, prompt shaping
and streaming, all litellm side. Mock the transport and assert those, so the
tests answer to our code instead of a vendor catalog.
2026-09-15 19:52:00 -07:00
Yuneng Jiang
a8fba14e10
test(proxy): assert budget resets decrement the cleared spend
The reset job moved from zeroing spend to an atomic decrement of the amount
it cleared, so every batched write now carries {"decrement": <cleared>}
instead of 0. Four tests still pinned 0 and had been failing since, which
also meant they no longer checked the amount at all. Assert the decrement
equals each row's own pre-reset spend, so a wrong amount fails the test.
2026-09-15 19:48:25 -07:00
yuneng-jiang
65d607d161
Merge pull request #41346 from BerriAI/litellm_e2e_provider_cache
feat(e2e): reuse exact provider responses for 24 hours
2026-09-15 19:45:56 -07:00
Yuneng Jiang
6fd988cc63
test(router): ignore deployment-selection logs in the fallback log assertion
simple_shuffle logs the selected deployment at INFO whenever a weight set
applies, so the fallback group's selection line lands between the fallback
notice and the success notice and pushed the notice out of the tail-3 window.
Filter it the same way the neighbouring get_available_deployment noise is
already filtered.
2026-09-15 19:45:33 -07:00
yucheng
f12feed9a9 test(proxy): expect litellm_call_id in the image generation call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 02:39:39 +00:00
yuneng-jiang
c64b821a7e
Merge pull request #41353 from BerriAI/litellm_/litellm-exception-oct-15-e61cb0
ci(image-scan): ignore zlib CVE-2026-85091 until Wolfi ships the fix
2026-09-15 19:32:53 -07:00
Yuneng Jiang
9170183087
fix(e2e): exclude unknown routes from upstream counters 2026-09-15 19:32:50 -07:00
Tin Chi Lo
39b7812810 feat(ui): configure capability and Fuse v2 classifiers 2026-09-15 19:30:50 -07:00
yucheng
1a7ca04cc5 fix(proxy): carry litellm_call_id through endpoint specific error logs and failure responses
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 02:29:26 +00:00
Yuneng Jiang
3dda798c8a
ci(e2e): consolidate cache contracts in filtered CircleCI job 2026-09-15 19:28:50 -07:00
yucheng-berri
c14ac1dbe9
Merge pull request #41329 from BerriAI/litellm_singulr_v2_contract 2026-09-15 19:27:30 -07:00
Yuneng Jiang
064d5d61da
ci(image-scan): ignore zlib CVE-2026-85091 until Wolfi ships the fix
Wolfi's security database names zlib 1.3.3-r0 as the fix for CVE-2026-85091,
but the newest zlib published to the Wolfi apk repo is 1.3.2-r7. Every
wolfi-base digest, including the current latest, still reports the CVE, so no
base image bump or apk upgrade can clear it and image-scan fails on every PR
touching a Dockerfile or the lockfile, and on the nightly schedule.

Ignore that CVE and its GHSA alias for the zlib apk package only, so a fixable
High in anything else still fails the job.
2026-09-15 19:23:21 -07:00
Joshua Valluru
84e14789d1 fix(mcp): preserve usable alternate header credentials 2026-09-15 19:18:48 -07:00
Yuneng Jiang
d94b9d117d
docs(e2e): clarify recorded IDs and quota header semantics 2026-09-15 19:16:45 -07:00
kerry-berri
399420bb25
Merge pull request #41339 from BerriAI/litellm_fix_fireworks_cost_components
fix(fireworks-ai): bill cache-write, reasoning and audio tokens via the shared cost calculator
2026-09-15 19:16:24 -07:00
Joshua Valluru
c7e4160ee6 fix(mcp): enforce OAuth write policy across signed callbacks 2026-09-15 19:15:34 -07:00
Yucheng He
146085669c fix(guardrails): run Singulr logging_only through the base hook, key MCP scans off the proxy call type and type the payloads
Removes the Singulr async_logging_hook and logging_hook overrides so logging_only runs through CustomGuardrail.async_logging_hook: the response scope reaches Singulr as an assistant message instead of a raw ModelResponse dump, a vendor timeout is recorded as guardrail_failed_to_respond, a request-scope block ends the scan, and the sync success callback thread makes no Singulr call.

Decides MCP versus LLM by the proxy logging object's call_type (then the call_type or server-only markers in request_data), never by name, arguments or mcp_tool_name keys a client can put in a chat body. REST /mcp-rest/tools/call pre-scans reach Singulr as mcp_request and a non-mapping arguments value is forwarded as tool_arguments instead of raising.

should_block is a strict bool defaulting to false so a null verdict is an invalid response that block_on_error decides; payload fields drop Any for Sequence, Mapping and AssistantMessage types; metadata carries only the keys present; docstrings and section comments removed per the repo comment policy.
2026-09-15 19:14:49 -07:00
aniket-kardile
8cce2b196a feat(guardrails): singulr v2 API contract with logging_only, pre_mcp_call and post_mcp_call
Squash of BerriAI/litellm#37464 (head da298ca7) by @aniket-kardile, adopted onto main: v2 gateway payload contract with request, response, mcp_request and mcp_response scopes, typed payload models, proxy user, org and team metadata forwarded to Singulr, and the logging_only, pre_mcp_call and post_mcp_call modes.
2026-09-15 19:14:49 -07:00
kerry
133f1e8ef5 test(fireworks-ai): drop the explanatory comment on the cache-read constant
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 02:06:42 +00:00