12363 commits

Author	SHA1	Message	Date
yucheng	d6f6f64c0f	fix(proxy): derive prompt injection heuristics thread count from CPU count with env override ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-17 05:30:41 +00:00
yucheng	719d7a1983	test(proxy): cover inherited moderation overrides through during_call_hook ... Replaces the capability flag assertion with a behavioral test that dispatches an async_moderation_hook inherited from a parent class, and drops the dispatch docstring that restated the code Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-17 05:00:01 +00:00
yucheng	2cbfd280e9	Merge remote-tracking branch 'origin/main' into litellm_prompt_injection_async_llm_check ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> # Conflicts: # litellm/proxy/proxy_server.py	2026-09-17 03:32:34 +00:00
tin-berri	d18e06f736	Merge pull request #41508 from BerriAI/litellm_1789600151_discover_context_limits ... feat(router): discover token limits for hosted OpenAI-compatible models	2026-09-16 20:29:57 -07:00
yucheng	3c000e4ffb	fix(proxy): run prompt injection heuristics on a dedicated bounded executor ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-17 02:38:13 +00:00
yucheng	d50bac391e	test(proxy): cover startup router wiring for registered prompt injection detectors ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-17 02:26:12 +00:00
yucheng	fc77914df3	test(proxy): type the moderation override stub in hook detection tests ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-17 01:59:04 +00:00
yucheng	b1255a6f2c	fix(proxy): run prompt injection heuristics off the event loop and dispatch llm_api_check moderation ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-17 01:33:54 +00:00
Yassin Kortam	351a54e849	Merge pull request #41507 from BerriAI/litellm_attribute_router_rejected_spend_provider ... fix(spend_tracking): attribute router-rejected requests to the model group provider	2026-09-16 18:21:44 -07:00
Mateo Wang	2445bdd2b5	Merge pull request #40934 from BerriAI/litellm_fix_ocr_native_multipage_pdf ... fix(logging): scan each log record once and collapse base64 payloads before the secret regex	2026-09-16 18:06:11 -07:00
yucheng-berri	0add8c0083	Merge pull request #41495 from BerriAI/litellm_converted_stream_post_call_hook ... fix(utils): run post-call deployment hook on converted chat streams	2026-09-16 17:49:14 -07:00
Yassin Kortam	617a40bb1c	Merge pull request #40842 from BerriAI/litellm_guardrail_tag_budget_enforcement ... fix(proxy): enforce tag budgets for tags added by guardrails	2026-09-16 17:44:29 -07:00
kerry-berri	db408f68ae	Merge pull request #41494 from BerriAI/litellm_auto_merge_price_sync ... ci: auto-merge provider-info-sync PRs when CI, Greptile and Bugbot are clean	2026-09-16 17:36:02 -07:00
yucheng	e1cce943de	Merge remote-tracking branch 'origin/main' into litellm_converted_stream_post_call_hook	2026-09-17 00:31:24 +00:00
mateo-berri	55cf4c43ed	fix(logging): stamp scrubbed records with a private sentinel a caller cannot supply ... A record stamped litellm_redacted=True skips the secret filter and both formatters, and extra={"litellm_redacted": True} on any log call put that stamp on a fresh record before the filter ran. The stamp is now a private object compared by identity, so only the filter's own pass marks a record scrubbed.	2026-09-16 17:30:49 -07:00
yassin	f76e8b3984	test(spend_tracking): type the provider resolution stubs in the router-rejected regression tests ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-17 00:29:13 +00:00
Moe Khalil	bab273ea0f	fix(router): preserve discovered limits and model info fallbacks ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-17 00:23:50 +00:00
mateo-berri	c340046de2	fix(logging): contain every extra serializer failure and skip rescanning a stamped record ... A pydantic model whose computed field raises escapes model_dump() and str() alike, and the secret filter caught only TypeError and ValueError, so the caller's own logger.warning() raised where the merge base contained the same failure inside the formatter. The scrub now catches every serializer failure and falls back to the object's text, or to the serializer's own "Unserializable Object" marker when even str() raises. JSON mode attaches the filter to uvicorn.error and the other third-party loggers and again to the root handler their records propagate to, so those records paid the secret regex twice. A record already stamped litellm_redacted now passes the filter untouched.	2026-09-16 17:22:12 -07:00
Mateo Wang	9e1eb546e4	Merge pull request #41514 from BerriAI/litellm_mcp_api_key_static_header_slot ... fix(mcp): count admin static headers as api_key credential slots	2026-09-16 17:22:08 -07:00
kerry-berri	2e4840ee18	Merge pull request #41509 from BerriAI/litellm_mantle_gpt5_verbosity ... fix(bedrock_mantle): accept and forward verbosity on gpt-5.x chat completions	2026-09-16 17:20:33 -07:00
Moe Khalil	d0ed8145b0	chore: merge latest main into model info discovery branch	2026-09-17 00:19:16 +00:00
Mateo Wang	d4a22acb66	Merge pull request #41513 from BerriAI/litellm_internal_copy_31400 ... fix(bedrock): neutralize orphaned tool blocks instead of raising or injecting a dummy tool (internal copy of #31400)	2026-09-16 17:12:35 -07:00
Moe Khalil	39286245b5	chore: merge main into model info discovery branch ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-17 00:10:00 +00:00
ryan-crabbe-berri	43713f7508	Merge pull request #39996 from BerriAI/litellm_team_admin_editable_fields ... feat(proxy): let proxy admins choose which team fields team admins may edit	2026-09-16 17:07:13 -07:00
kerry	0a47fe160d	Merge remote-tracking branch 'origin/main' into litellm_mantle_gpt5_verbosity	2026-09-17 00:06:50 +00:00
Mateo Wang	09a188b583	Merge pull request #41094 from BerriAI/litellm_model_group_info_proxy_admin_all_models ... fix(proxy): show all model groups to proxy admins in /model_group/info	2026-09-16 17:06:44 -07:00
Mateo Wang	913ef6ed49	Merge pull request #33856 from BerriAI/litellm_azure_ai_responses_native ... fix(azure_ai): route Responses API to native /openai/v1/responses for Foundry Models	2026-09-16 17:06:29 -07:00
kerry-berri	f93b31679b	Merge pull request #41503 from BerriAI/litellm_fix_interrupted_anthropic_reasoning_usage ... fix(streaming): estimate interrupted Anthropic stream usage from reasoning_content	2026-09-16 17:04:59 -07:00
yassin	b35ca7d2c3	fix(spend_tracking): honour the global litellm_proxy override when inferring a model group provider ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-17 00:03:01 +00:00
Mateo Wang	6cdf398bea	Merge pull request #41504 from BerriAI/litellm_bedrock_agent_runtime_strip_virtual_key ... fix(proxy): stop forwarding LiteLLM credential headers on Bedrock agent-runtime passthrough	2026-09-16 17:02:17 -07:00
kerry	a3d3fe4ada	fix(ci): pin the auto-merge request to the evaluated head sha ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-16 23:57:44 +00:00
kerry	42bf9f0ece	Merge remote-tracking branch 'origin/main' into litellm_auto_merge_price_sync	2026-09-16 23:56:48 +00:00
yassin	d6b13f938d	test(proxy): cover guardrail tag budget edge cases ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-16 23:52:39 +00:00
yassin	488666ccae	fix(proxy): enforce tag budgets for tags added by guardrails ... Auth runs the tag budget check before pre_call_hook, so a tag that a custom guardrail adds is attributed spend but never budget checked. After the pre-call hook, budget check only the newly added tags with the same exemptions auth applied (budget-free routes, zero-cost models), keep the pre-guardrail tag baseline across fallback retries, and surface an over-budget tag as the same budget_exceeded 429 auth returns Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-16 23:52:39 +00:00
Mateo Wang	319f427c40	Merge pull request #41201 from BerriAI/litellm_gemini_37_38_flash_no_minimal_thinking ... fix(gemini): map minimal thinking to low for Gemini 3.7 and 3.8 Flash	2026-09-16 16:52:08 -07:00
yassin	5f1d87911a	test(spend_tracking): cover unresolvable deployment leaving provider empty ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-16 23:50:22 +00:00
mateo-berri	ada0a1ad3a	fix(azure_ai): strip the azure_ai/ prefix when a Responses call is remapped to azure ... A catalog OpenAI name on an .openai.azure.com host (or with AZURE_AI_API_BASE set to one) is remapped from azure_ai to azure before the Responses request is built, and the azure_ai/ prefix stayed in the wire model, so Azure answered DeploymentNotFound. The Azure Responses config now strips azure_ai/ next to responses/ and o_series/.	2026-09-16 16:49:29 -07:00
yucheng-berri	c5325b1492	Merge pull request #40596 from BerriAI/litellm_lit_7470_rate_limit_fallback_pristine_data ... fix(proxy): retry rate-limit fallbacks from a pristine request snapshot	2026-09-16 16:47:35 -07:00
yucheng-berri	672f43fd54	Merge pull request #41356 from BerriAI/litellm_lit7836_call_id_endpoint_logs ... fix(proxy): carry litellm_call_id through endpoint specific error logs and failure responses	2026-09-16 16:43:12 -07:00
mateo-berri	574ea15b8f	fix(mcp): count admin static headers as api_key credential slots ... An api_key server whose key lives in static_headers, the documented shape for upstreams that expect a custom header name, dispatched fine before the fail-closed check and was rejected as misconfigured after it. The check now treats every static header the admin configured as a credential slot for api_key mode, on both the MCP client path and the OpenAPI tool path, with regression tests at all three layers.	2026-09-16 16:42:20 -07:00
mateo-berri	41737aeda8	test(proxy): drop the docstring from the agent-runtime passthrough regression class	2026-09-16 16:37:32 -07:00
Moe Khalil	b95dbb41ae	fix(router): scope discovered limits to active deployments ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-16 23:37:28 +00:00
yassin	9af014d75a	fix(spend_tracking): keep inferred provider out of model reconstruction and OAuth provider lookup ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-16 23:36:54 +00:00
kerry	e8f246bb6b	fix(responses_bridge): forward verbosity as text.verbosity ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-16 23:32:32 +00:00
Mateo Wang	390ab45448	Merge pull request #37506 from BerriAI/litellm_dashscope_reasoning_effort ... fix(dashscope): forward reasoning_effort to the provider	2026-09-16 16:29:25 -07:00
mateo-berri	2f186055e6	fix(logging): decide keep-or-scrub for a log extra by comparing it to its scrubbed copy ... The code-quality check refuses recursive functions and the walk that inspected extras was one, so the filter no longer walks anything itself. safe_dumps now builds its JSON-native structure through safe_json_structure, the filter scrubs the extra through that, and the original object is kept only when the scrubbed copy compares equal to it. Anything the serializer skipped (non-string keys, nests past its depth, fields a repr hides) makes the copy differ, so the copy wins. A host object whose equality raises, as numpy arrays and torch tensors do, counts as changed instead of breaking the caller's log call	2026-09-16 16:29:23 -07:00
kerry	884e96958c	fix(streaming): cap estimated reasoning tokens to the provider total and cover dict chunks ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-16 23:27:48 +00:00
Moe Khalil	1c9525fdab	test(router): cover deployment replacement during metadata discovery ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-16 23:27:31 +00:00
kerry	15c18ad6cd	fix(bedrock_mantle): accept verbosity on gpt-5.x chat completions ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-16 23:27:07 +00:00
joshua-berri	41410e9556	Merge pull request #41364 from BerriAI/litellm_fix_mcp_auth_fail_closed_4501 ... fix(mcp): fail closed on missing upstream credentials	2026-09-16 23:26:38 +00:00