scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-22 00:31:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 51
49660 commits 2566 branches 1413 tags 3.2 GiB
Commit graph

12067 commits

Author SHA1 Message Date
yucheng
05ededf8a0 fix(otel): parent passthrough trace propagation on the legacy request span 
Pass user_api_key_dict.parent_otel_span into the outgoing W3C injection so the
legacy otel callback propagates its litellm_request span, falling back to the
otel_v2 request root span and then the ambient span. Extend the mapped unit
tests to assert the propagated trace and span ids over real captured headers
for HTTP and WebSocket passthrough with forwarding on and off.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 07:24:52 +00:00
shivam
7ba073aa26 test(otel): cover websocket trace propagation with forwarding on and off 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 06:55:06 +00:00
shivam
68cc12e848 test(otel): assert websocket forwarded header reaches upstream 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 06:55:06 +00:00
shivam
e4a12510f6 test(otel): cover websocket forwarded headers 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 06:55:06 +00:00
shivam
d2e8b9c656 fix(otel): keep passthrough working when opentelemetry is not installed 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 06:55:06 +00:00
shivam
7bce15f7d8 fix(otel): propagate W3C trace context on passthrough upstream requests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 06:55:06 +00:00
kerry-berri
5960881640
Merge pull request #41337 from BerriAI/litellm_fix_responses_stream_absent_usage_recount 
fix(responses): recount tokens when a streamed response completes without usage
 2026-09-15 21:19:42 -07:00
kerry
ff878e7df0 refactor(responses): copy the terminal event instead of mutating stubbed chunks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 03:54:04 +00:00
kerry
7cc07d437a fix(responses): build the billed terminal response immutably and guard the cache dump 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 03:47:03 +00:00
kerry-berri
4b84fa9230
Merge pull request #41336 from BerriAI/litellm_fix_anthropic_stream_absent_usage 
fix(anthropic): tolerate message_delta events without usage when streaming
 2026-09-15 20:23:01 -07:00
kerry
7121e64db4 fix(responses): type the dict terminal response so the estimated usage is billed 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 03:13:53 +00:00
Yassin Kortam
d108cdc431
Merge pull request #38254 from BerriAI/litellm_fix_xai_responses_instructions 2026-09-15 19:55:52 -07:00
yuneng-jiang
65d607d161
Merge pull request #41346 from BerriAI/litellm_e2e_provider_cache 
feat(e2e): reuse exact provider responses for 24 hours
 2026-09-15 19:45:56 -07:00
Yuneng Jiang
3dda798c8a
ci(e2e): consolidate cache contracts in filtered CircleCI job 2026-09-15 19:28:50 -07:00
yucheng-berri
c14ac1dbe9
Merge pull request #41329 from BerriAI/litellm_singulr_v2_contract 2026-09-15 19:27:30 -07:00
kerry-berri
399420bb25
Merge pull request #41339 from BerriAI/litellm_fix_fireworks_cost_components 
fix(fireworks-ai): bill cache-write, reasoning and audio tokens via the shared cost calculator
 2026-09-15 19:16:24 -07:00
Yucheng He
146085669c fix(guardrails): run Singulr logging_only through the base hook, key MCP scans off the proxy call type and type the payloads 
Removes the Singulr async_logging_hook and logging_hook overrides so logging_only runs through CustomGuardrail.async_logging_hook: the response scope reaches Singulr as an assistant message instead of a raw ModelResponse dump, a vendor timeout is recorded as guardrail_failed_to_respond, a request-scope block ends the scan, and the sync success callback thread makes no Singulr call.

Decides MCP versus LLM by the proxy logging object's call_type (then the call_type or server-only markers in request_data), never by name, arguments or mcp_tool_name keys a client can put in a chat body. REST /mcp-rest/tools/call pre-scans reach Singulr as mcp_request and a non-mapping arguments value is forwarded as tool_arguments instead of raising.

should_block is a strict bool defaulting to false so a null verdict is an invalid response that block_on_error decides; payload fields drop Any for Sequence, Mapping and AssistantMessage types; metadata carries only the keys present; docstrings and section comments removed per the repo comment policy.
 2026-09-15 19:14:49 -07:00
aniket-kardile
8cce2b196a feat(guardrails): singulr v2 API contract with logging_only, pre_mcp_call and post_mcp_call 
Squash of BerriAI/litellm#37464 (head da298ca7) by @aniket-kardile, adopted onto main: v2 gateway payload contract with request, response, mcp_request and mcp_response scopes, typed payload models, proxy user, org and team metadata forwarded to Singulr, and the logging_only, pre_mcp_call and post_mcp_call modes.
 2026-09-15 19:14:49 -07:00
kerry
133f1e8ef5 test(fireworks-ai): drop the explanatory comment on the cache-read constant 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 02:06:42 +00:00
kerry
7680d3de86 fix(responses): tolerate dict terminal responses when estimating usage 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 02:05:44 +00:00
kerry-berri
54e1b9e3de
Merge pull request #41338 from BerriAI/litellm_fix_gemini_model_version 
fix(gemini): propagate the provider's modelVersion to the response model
 2026-09-15 19:04:50 -07:00
yucheng-berri
41eb2dbfeb
Merge pull request #41128 from BerriAI/litellm_llm_judge_pre_call 
feat(guardrails): support pre_call and during_call modes for llm_as_a_judge
 2026-09-15 18:58:21 -07:00
tin-berri
106365fbac
Merge pull request #41341 from BerriAI/litellm_autorouter_pricing_fields 
fix(proxy): preserve Anthropic pricing modifiers in router savings
 2026-09-15 18:56:15 -07:00
kerry
7ed20406d7 test(responses): narrow the ValueError assertion to satisfy PT011 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:47:45 +00:00
Yassin Kortam
67cb0bc089
Merge pull request #41313 from BerriAI/litellm_model_activity_response_time 
feat(ui): show average response time per model in usage model activity
 2026-09-15 18:43:48 -07:00
kerry
091a38cce6 test(anthropic): import Final for the annotated locals 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:43:08 +00:00
kerry
c4c96180e3 fix(gemini): strip version suffix from modelVersion and keep it on blocked streams 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:42:36 +00:00
kerry
8352045f32 style(fireworks-ai): apply repository conventions to the cost component change 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:39:24 +00:00
Devin AI
8d625d0400 fix(xai): keep 'instructions' on the xAI Responses API so system messages survive web_search bridging 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:35:13 +00:00
kerry
10aef224e7 style(anthropic): apply repository conventions to the missing-usage change 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:34:28 +00:00
kerry
d1aa37b0a8 merge main into litellm_fix_fireworks_cost_components 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:28:29 +00:00
kerry-berri
171888b716
Merge pull request #41298 from BerriAI/litellm_drop_remaining_vendor_fact_pins 
test: drop remaining tests that pin cost-map vendor facts
 2026-09-15 18:25:14 -07:00
kerry
a6d2332f7a test(responses): drive the usage-estimate failure path without patching litellm 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:24:35 +00:00
kerry
5a9fe56aff fix(responses): count custom-tool and MCP argument deltas in the streamed usage fallback 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:24:05 +00:00
kerry
44a6d19889 test(gemini): drop review narration from the wrapper test docstring 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:14:37 +00:00
kerry
1484fd7600 fix(responses): keep the usage estimate best-effort when token counting raises 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:13:26 +00:00
yucheng
f66ffc387e fix(guardrails): only honor the judge call-origin stamp on logging_only in llm_as_a_judge 
On pre_call, during_call and post_call the hook data is the client request body, so a
client-supplied litellm_params.metadata.internal_call_origin must not skip enforcement.
Type the during_call helper's request payload as dict[str, object]

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
kerry
801f6a92b4 test(gemini): assert the served modelVersion reaches the assembled stream through CustomStreamWrapper 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
dd4829398a fix(guardrails): log the configured mode when logging_only is mixed with an enforcing llm_as_a_judge mode 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
10506ab904 fix(guardrails): judge the whole latest user turn, run every during_call guardrail, log combined modes as configured 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
1b8b17cc8a fix(guardrails): judge only the latest request turn in pre_call and during_call llm_as_a_judge 
The request-side prompt told the judge to focus on the most recent user turn but the text under review was every extracted request message joined together, so a multi-turn request with an off-topic earlier turn and an on-topic latest turn scored 50 and was blocked. Request-side judging now evaluates the last extracted request text (after the configured message scoping) and passes the full role-labelled conversation only as context. Response-side judging still evaluates all extracted response text

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
e9357d9a6f fix(guardrails): stop logging_only llm_as_a_judge from judging its own judge calls 
Judge sub-calls now carry the internal_call_origin metadata stamp and the
guardrail skips any logged call bearing it, so a logging_only judge no longer
recurses into an unbounded chain of judge requests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
a658f20005 test(guardrails): grant premium_user for the tagged Mode case in llm_as_a_judge mode-shape test 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
4d1330ea0b test(guardrails): drop callback-manager patch from llm_as_a_judge mode-shape test 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
28bd00a004 fix(guardrails): label llm_as_a_judge logging_only verdicts with their mode 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
b93f22265c fix(guardrails): give the request-side judge role-labelled conversation context 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
6fccf00d93 fix(guardrails): keep list and tagged mode shapes for llm_as_a_judge 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:52 +00:00
yucheng
0b34800501 fix(guardrails): resolve llm_as_a_judge request-side log mode from event_hook lists, drop unused alias 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:52 +00:00
yucheng
77a6327675 feat(guardrails): support pre_call and during_call modes for llm_as_a_judge 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:52 +00:00
kerry
de9aa48cd6 fix(responses): count multimodal input and tool-call output in the streamed usage fallback 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:02:29 +00:00