scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 35
litellm/litellm
History
Exact
Tin Chi Lo 2d5ffc56cc feat(shadow_eval): measure both arms' cost so a job reports what the router would have saved 
The attempt row now prices the real arm (the payload's response_cost plus its own
routing classifier when it routed) beside the shadow arm (completion plus the
classifier cost the routing decision writes back), and flags turns litellm's
response cache served. A per-leg funnel table counts the eligible requests that
produced no row (lost the sampling dice, unjudgeable shape, concurrency shed),
so results can weigh judged rows against the traffic they stand for. Job results
gain per-slice and overall arm spends plus the coverage counts, the budget gates
charge the shadow arm's classifier spend against max_budget, and the dashboard
shows the measured cost comparison beside the win rate

Resolves LIT-6358

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 21:45:25 -07:00
..
a2a_protocol fix(a2a): speak the 0.3 dialect to servers with mis-cased protocol bindings 2026-08-24 10:51:14 -07:00
anthropic_interface
assistants
batch_completion
batches chore(typing): tighten annotations added in the last day and ratchet budgets 2026-08-23 07:49:54 +00:00
caching fix(caching): treat an empty cache namespace as no namespace 2026-08-26 13:48:45 -07:00
completion_extras fix(anthropic): carry tool_reference tool results through the guardrail translation round trip 2026-08-26 22:35:46 -07:00
compression
containers chore(typing): drop 1.3k basedpyright errors across 30 Any hotspot files 2026-08-16 03:56:02 +00:00
endpoints/speech/speech_to_completion_bridge fix(speech): keep proxy metadata and completion cost through the TTS completion bridge 2026-08-26 15:12:41 -07:00
evals chore(typing): remove 914 basedpyright Any errors across 16 hotspot files 2026-08-10 01:24:40 -07:00
experimental_mcp_client feat(mcp): let a resolved OAuth token target a custom upstream header (#38456) 2026-08-27 14:32:01 -07:00
files fix(batches): accept litellm_proxy in files and batches provider type literals 2026-08-18 13:28:06 -07:00
fine_tuning
google_genai refactor(google_genai): pick the stream logging endpoint type at construction 2026-08-26 17:18:44 -07:00
images Merge origin/litellm_internal_staging into litellm_fix_36493_image_video_routes 2026-08-24 11:58:22 -07:00
integrations feat(shadow_eval): measure both arms' cost so a job reports what the router would have saved 2026-08-27 21:45:25 -07:00
interactions feat(vertex_ai): add native Vertex AI Interactions API support 2026-08-25 09:55:13 -07:00
litellm_core_utils fix(shadow_eval): refuse a judge model that also serves one of the arms it grades (#38589) 2026-08-27 18:44:44 -07:00
llms fix(moonshot, together_ai): send the reasoning effort Kimi K3 accepts (#38611) 2026-08-27 20:46:46 -07:00
models refactor(repositories): type prisma table access with one generic protocol 2026-08-25 12:14:17 +00:00
ocr fix(ocr): reject invalid req_format values as 400 on the SDK path 2026-08-17 15:49:18 -07:00
passthrough fix(bedrock_mantle): register a Bedrock runtime passthrough config so /bedrock/model/<deployment>/invoke works 2026-08-25 10:10:38 -07:00
proxy feat(shadow_eval): measure both arms' cost so a job reports what the router would have saved 2026-08-27 21:45:25 -07:00
proxy_auth
rag chore(typing): remove 914 basedpyright Any errors across 16 hotspot files 2026-08-10 01:24:40 -07:00
realtime_api fix(health): probe Azure GA realtime path for transcription-only models 2026-08-26 12:14:52 -07:00
repositories feat(proxy): opt-in budget rollover carrying overage into the next window (#38514) 2026-08-27 12:46:09 -07:00
rerank_api fix(together_ai): default endpoints to api.together.ai instead of api.together.xyz 2026-08-25 10:18:55 -07:00
responses fix(realtime): carry audio output tokens into response.done usage so Gemini Live native audio bills at the audio rate 2026-08-26 21:10:12 -07:00
router_strategy fix(router): reject complexity-router settings written outside complexity_router_config (#38570) 2026-08-27 17:04:49 -07:00
router_utils fix(moonshot, together_ai): send the reasoning effort Kimi K3 accepts (#38611) 2026-08-27 20:46:46 -07:00
rust_bridge feat(rust): route /chat/completions through the Rust core for anthropic and bedrock (#37241) 2026-08-20 16:15:24 -07:00
sandbox
search
secret_managers fix(azure): prefer workload identity over managed identity 2026-08-20 13:45:56 -07:00
skills
types feat(shadow_eval): measure both arms' cost so a job reports what the router would have saved 2026-08-27 21:45:25 -07:00
vector_store_files
vector_stores refactor(repositories): type prisma table access with one generic protocol 2026-08-25 12:14:17 +00:00
videos refactor(videos): make video upload param keyword-only on public edit fns 2026-08-24 15:55:07 -07:00
__init__.py feat(proxy): opt-in budget rollover carrying overage into the next window (#38514) 2026-08-27 12:46:09 -07:00
_internal_context.py
_lazy_imports.py
_lazy_imports_registry.py fix(together_ai): route chat completions through a dedicated TogetherAIChatConfig 2026-08-25 11:44:06 -07:00
_logging.py fix(logging): stop stream-based log collectors classifying INFO logs as errors (#38476) 2026-08-27 17:42:00 -07:00
_redis.py refactor(redis): drop docstrings restating the code 2026-08-26 10:45:56 -07:00
_redis_credential_provider.py
_service_logger.py
_uuid.py
_version.py
anthropic_beta_headers_config.json
anthropic_beta_headers_manager.py
blog_posts.json
budget_manager.py
constants.py fix(presidio): chunk oversized text before /analyze so large content blocks do not fail (#38483) 2026-08-27 17:41:55 -07:00
cost.json
cost_calculator.py Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_together_cache_pricing 2026-08-27 14:29:46 -07:00
exceptions.py feat(proxy): redact or drop individual batch records instead of rejecting the file (#37561) 2026-08-20 13:12:55 -07:00
main.py fix: suppress misleading register_model unresolved-cost warnings for entries without custom pricing (#38542) 2026-08-27 12:45:30 -07:00
model_prices_and_context_window_backup.json Merge pull request #38100 from FelipeRodriguesGare/bugfix/tencent-thinking-extra-body 2026-08-27 18:23:11 -07:00
policy_templates_backup.json
provider_endpoints_support_backup.json fix(scx-ai): use the published scx.ai rates and the scx_ai docs url 2026-08-21 17:28:12 +10:00
py.typed
router.py Merge pull request #38606 from BerriAI/litellm_bedrock_messages_midstream_fallback 2026-08-27 19:13:18 -07:00
scheduler.py
setup_wizard.py
timeout.py
utils.py feat(model_prices): let a map entry declare its exact reasoning_effort levels (#38481) 2026-08-27 15:38:01 -07:00