scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-17 23:51:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 67
49597 commits 2515 branches 1410 tags 3 GiB
Commit graph

5785 commits

Author SHA1 Message Date
yucheng-berri
41eb2dbfeb
Merge pull request #41128 from BerriAI/litellm_llm_judge_pre_call 
feat(guardrails): support pre_call and during_call modes for llm_as_a_judge
 2026-09-15 18:58:21 -07:00
tin-berri
106365fbac
Merge pull request #41341 from BerriAI/litellm_autorouter_pricing_fields 
fix(proxy): preserve Anthropic pricing modifiers in router savings
 2026-09-15 18:56:15 -07:00
Yassin Kortam
67cb0bc089
Merge pull request #41313 from BerriAI/litellm_model_activity_response_time 
feat(ui): show average response time per model in usage model activity
 2026-09-15 18:43:48 -07:00
kerry-berri
171888b716
Merge pull request #41298 from BerriAI/litellm_drop_remaining_vendor_fact_pins 
test: drop remaining tests that pin cost-map vendor facts
 2026-09-15 18:25:14 -07:00
yucheng
f66ffc387e fix(guardrails): only honor the judge call-origin stamp on logging_only in llm_as_a_judge 
On pre_call, during_call and post_call the hook data is the client request body, so a
client-supplied litellm_params.metadata.internal_call_origin must not skip enforcement.
Type the during_call helper's request payload as dict[str, object]

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
dd4829398a fix(guardrails): log the configured mode when logging_only is mixed with an enforcing llm_as_a_judge mode 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
10506ab904 fix(guardrails): judge the whole latest user turn, run every during_call guardrail, log combined modes as configured 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
1b8b17cc8a fix(guardrails): judge only the latest request turn in pre_call and during_call llm_as_a_judge 
The request-side prompt told the judge to focus on the most recent user turn but the text under review was every extracted request message joined together, so a multi-turn request with an off-topic earlier turn and an on-topic latest turn scored 50 and was blocked. Request-side judging now evaluates the last extracted request text (after the configured message scoping) and passes the full role-labelled conversation only as context. Response-side judging still evaluates all extracted response text

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
e9357d9a6f fix(guardrails): stop logging_only llm_as_a_judge from judging its own judge calls 
Judge sub-calls now carry the internal_call_origin metadata stamp and the
guardrail skips any logged call bearing it, so a logging_only judge no longer
recurses into an unbounded chain of judge requests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
a658f20005 test(guardrails): grant premium_user for the tagged Mode case in llm_as_a_judge mode-shape test 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
4d1330ea0b test(guardrails): drop callback-manager patch from llm_as_a_judge mode-shape test 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
28bd00a004 fix(guardrails): label llm_as_a_judge logging_only verdicts with their mode 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
b93f22265c fix(guardrails): give the request-side judge role-labelled conversation context 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:53 +00:00
yucheng
6fccf00d93 fix(guardrails): keep list and tagged mode shapes for llm_as_a_judge 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:52 +00:00
yucheng
0b34800501 fix(guardrails): resolve llm_as_a_judge request-side log mode from event_hook lists, drop unused alias 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:52 +00:00
yucheng
77a6327675 feat(guardrails): support pre_call and during_call modes for llm_as_a_judge 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 01:05:52 +00:00
Tin Chi Lo
5aa2adbacd fix(proxy): preserve Anthropic pricing modifiers in router savings 2026-09-15 18:02:05 -07:00
yucheng
9bd3f7b885 Merge remote-tracking branch 'origin/main' into litellm_agent365_mcp_guardrail 2026-09-16 00:52:27 +00:00
yucheng-berri
0b3e56448f
Merge pull request #41126 from BerriAI/litellm_custom_code_guardrail_identity 
fix(guardrails): resolve caller identity from metadata buckets in custom code guardrail
 2026-09-15 17:47:17 -07:00
tin-berri
878716f806
Merge pull request #41326 from BerriAI/litellm_forecast_classifier_entitlement 
feat(router): limit unlicensed Capability and Fuse v2 routers to one each
 2026-09-15 17:36:22 -07:00
kerry
ba6bcd747f chore: merge origin/main into test cleanup 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 00:33:25 +00:00
Mateo Wang
aa710dc6a9
Merge pull request #35388 from BerriAI/litellm_session_total_duration 
fix(spend): sum multi-round session duration in logs UI
 2026-09-15 17:31:41 -07:00
Yassin Kortam
7eeba69016
Merge pull request #41316 from BerriAI/litellm_nvidia_nim_infer_passthrough 
feat(proxy): add /nvidia_nim passthrough route for NIM object detection and OCR /v1/infer
 2026-09-15 17:28:25 -07:00
Yassin Kortam
5071d96d16
Merge pull request #41255 from BerriAI/litellm_org_member_spend_tracking 
fix(proxy): track per-member organization spend
 2026-09-15 17:25:40 -07:00
Yassin Kortam
e4a7d2aa0b
Merge pull request #41302 from BerriAI/litellm_fix_key_model_rpm_override_precedence 
fix(proxy): key model rpm/tpm override takes precedence over team model limit
 2026-09-15 17:24:38 -07:00
Yassin Kortam
3413efcd7f
Merge pull request #41306 from BerriAI/litellm_lit2435_litellm_log_error_silences_info 
fix(proxy): honor LITELLM_LOG for uvicorn and proxy extras loggers
 2026-09-15 17:22:39 -07:00
Yassin Kortam
636313e9fc
Merge pull request #41322 from BerriAI/litellm_gcp_video_usage 
fix(vertex_ai): bill Gemini Omni Interactions usage and Veo sampleCount on passthrough
 2026-09-15 17:21:15 -07:00
yassin
65f9d9bbbd test(proxy): use the module-level HTTPException import in the /nvidia_nim route tests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 23:57:16 +00:00
Yassin Kortam
24153b5f29
Merge pull request #41308 from BerriAI/litellm_resolve_model_group_alias_before_auth 2026-09-15 16:51:28 -07:00
Yassin Kortam
474563a4ba
Merge pull request #41303 from BerriAI/litellm_passthrough_auth_false_db_overlay 2026-09-15 16:50:41 -07:00
yucheng-berri
c1a3784e6c
Merge pull request #41205 from BerriAI/litellm_lit_5856_error_log_call_id 
fix(proxy): include litellm_call_id in LLM API exception logs
 2026-09-15 16:47:57 -07:00
yassin
74a2c2eab1 refactor(vertex_ai): tighten Interactions route match and type the usage boundary 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 23:34:31 +00:00
yassin
771b2509d1 fix(proxy): reject mixed NIM model groups and strip the deployment model before the group in /nvidia_nim URLs 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 23:34:22 +00:00
mateo-berri
3441f73118 Merge remote-tracking branch 'origin/main' into litellm_session_total_duration 
# Conflicts:
#	litellm/proxy/spend_tracking/spend_management_endpoints.py
#	tests/test_litellm/proxy/spend_tracking/test_spend_management_endpoints.py
#	ui/litellm-dashboard/src/components/view_logs/RequestLogsTableColumns.tsx
#	ui/litellm-dashboard/src/components/view_logs/columns.tsx
 2026-09-15 16:17:18 -07:00
Tin Chi Lo
39cf1f302d feat(router): apply entitlement limits to forecast classifiers 2026-09-15 16:17:00 -07:00
yassin
c007fb9928 fix(proxy): skip alias rewrite only for dispatched pass-through handlers 
Match the pass-through skip to what FastAPI actually dispatched (the user-defined
endpoint marker or a provider handler's {endpoint:path} param) instead of the
mapped route prefixes, which also cover native routes such as /openai/v1/responses
and /cursor/chat/completions. Wrap the added test lines to the 120-column limit.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 23:00:08 +00:00
Mateo Wang
6bc821492b
Merge pull request #40476 from BerriAI/litellm_codex_model_catalog_sync 
feat(cli): sync Codex /model picker from proxy /v1/models in lite codex
 2026-09-15 15:59:04 -07:00
Yassin Kortam
9d75cdd502
Merge pull request #41304 from BerriAI/litellm_lit1698_openai_system_messages_first 2026-09-15 15:56:13 -07:00
yassin
b50a22b370 fix(proxy): restrict /nvidia_nim route to NIM-backed model groups and inject router in tests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:55:14 +00:00
Yassin Kortam
9bb83fdaea
Merge pull request #41281 from BerriAI/litellm_lit_7417_jwt_key_mapping_issuer_scope 
fix(jwt-auth): scope JWT key mappings by issuer to prevent cross-issuer collisions
 2026-09-15 15:51:50 -07:00
yassin
fd90eeb3c6 test(proxy): cover disjoint-method db/yaml pass-through entries on a shared path 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:50:29 +00:00
Yassin Kortam
94a6dfaf52
Merge pull request #41307 from BerriAI/litellm_passthrough_deployment_model_id 
fix(passthrough): attribute Vertex passthrough successes to the resolved router deployment
 2026-09-15 15:44:38 -07:00
yassin
b13cf21fd1 fix(vertex_ai): bill Gemini Omni Interactions usage and Veo sampleCount on passthrough 
Route Vertex and Gemini /interactions create responses through the shared
Interactions usage transformation so SpendLogs carry per-modality tokens and
cost instead of zeros, and carry parameters.sampleCount into Veo usage as
video_count so duration-based video cost is multiplied by the number of
generated videos

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:42:27 +00:00
kerry
357e0fca8e chore: merge origin/main into test cleanup 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:36:58 +00:00
yassin
db0f06d153 fix(proxy): skip team model tpm accounting when key owns model tpm limit 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:35:48 +00:00
yassin
c48723f210 fix(proxy): let a db pass-through entry override the yaml entry on the same path 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:34:35 +00:00
yassin
5d6e367d56 fix(proxy): keep alias rewrite off pass-through bodies and auth-merged params 
Skip router_settings.model_group_alias resolution on registered pass-through
routes, rebuild the rewritten body from the cached client payload instead of the
auth-enriched request_data, and centralize the resolved-scope sentinel in
litellm/constants.py

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:32:11 +00:00
yassin
1b6e76af16 fix(proxy): resolve LITELLM_LOG for uvicorn at startup and restore logger state in tests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:27:24 +00:00
yassin
5b0fa89056 feat(ui): show average response time per model in usage model activity 
Roll request_duration_ms of successful, non-internal requests into the
daily spend tables as total_response_time_ms plus timed_requests, expose
both through the daily activity endpoints, and derive the average in the
Usage -> Model Activity view of the Admin UI

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:16:12 +00:00
kerry
2586b21893 test: keep tests that survive correct cost-map updates 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:16:08 +00:00