50181 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
mateo-berri	d26e007229	Merge remote-tracking branch 'origin/main' into litellm_responses_per_message_guardrail_rewrite	2026-09-13 01:05:36 -07:00
mateo-berri	09314f239c	fix(guardrails): hand per-message rewrites back as structured_messages ... A guardrail that rewrites text per chat message now returns the rewritten rows as structured_messages instead of only texts, so the Responses and chat handlers write the rewrite back through the structured path. The generic guardrail API response accepts an optional structured_messages list, Prompt Security modify builds one from modified_messages, and rows a server echoes back exactly as shown are restored to the original row objects because the request model drops undeclared keys. Texts-only per-message answers keep the named rejection on both endpoints.	2026-09-13 00:54:29 -07:00
mateo-berri	4d2352d0b5	fix(cost): bill per-query priced rerank deployments from their router model id	2026-09-13 00:45:33 -07:00
yucheng	a2501d3b59	chore: re-run review bots on the reopened PR tip ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-13 07:06:56 +00:00
mateo-berri	af363fc29c	Merge remote-tracking branch 'origin/main' into litellm_lit4995_vertex_rerank_search_units	2026-09-13 00:03:04 -07:00
mateo-berri	4c022a3089	feat(pricing): add azure gpt-chat-latest global and data zone rates	2026-09-12 23:34:12 -07:00
yucheng	d7623112fa	fix(mcp): hand Agent 365 the admin description override and clear the LIT001 and schema gates ... get_listed_tool now applies tool_name_to_description so the guardrail evaluates the description clients saw, with a regression test. The listed-tools cache and tool_input_schema fields use Mapping annotations so the change adds no LIT001 findings, and schema.d.ts is regenerated with npm run gen:api Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-13 05:15:33 +00:00
yassin	e0cf63c5fa	Merge remote-tracking branch 'origin/main' into litellm_per_issuer_virtual_key_claim_field	2026-09-13 05:00:54 +00:00
berriai-litellm-provider-info-sync[bot]	6423acc11a	chore(prices): sync prices for 5 providers: 278 models, 34 new ... fireworks_ai/accounts/fireworks/models/deepseek-v4-flash-0731: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/deepseek-v4-flash-0731: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/accounts/fireworks/models/deepseek-v4-flash-vision-exp: fireworks_ai/deepseek-v4-flash-vision-exp: fireworks_ai/accounts/fireworks/models/deepseek-v4-pro: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost, input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/deepseek-v4-pro: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost, input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/accounts/fireworks/models/deepseek-v4-pro-0813: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/accounts/fireworks/models/deepseek-v4p1-flash: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/deepseek-v4p1-flash: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/accounts/fireworks/models/glm-5p2: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/glm-5p2: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/accounts/fireworks/models/glm-5p3: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/accounts/fireworks/models/glm-5p3-flash: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/accounts/fireworks/models/gpt-oss-120b: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/gpt-oss-120b: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/accounts/fireworks/models/kimi-k2p6: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/kimi-k2p6: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/accounts/fireworks/models/kimi-k2p7-code: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/kimi-k2p7-code: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/accounts/fireworks/models/kimi-k3: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/kimi-k3: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/accounts/fireworks/models/minimax-m2p7: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/minimax-m2p7: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/accounts/fireworks/models/minimax-m3: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/minimax-m3: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/accounts/fireworks/models/muse-glimmer-30b: fireworks_ai/muse-glimmer-30b: fireworks_ai/accounts/fireworks/models/nemotron-3-ultra-nvfp4: fireworks_ai/nemotron-3-ultra-nvfp4: fireworks_ai/accounts/fireworks/models/qwen3-embedding-8b: fireworks_ai/accounts/fireworks/models/qwen3-reranker-8b: input_cost_per_token fireworks_ai/accounts/fireworks/models/qwen3p7-plus: fireworks_ai/qwen3p7-plus: fireworks_ai/accounts/fireworks/models/qwen3p8-max: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/qwen3p8-max: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority fireworks_ai/accounts/fireworks/routers/glm-5p2-fast: fireworks_ai/accounts/fireworks/routers/glm-5p3-fast: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost fireworks_ai/accounts/fireworks/routers/kimi-k3-fast: together_ai/arcee-ai/trinity-mini: input_cost_per_token, output_cost_per_token together_ai/arize-ai/qwen-2-1.5b-instruct: babbage-002: input_cost_per_token_batches, output_cost_per_token_batches chat-latest: chatgpt-image-latest: output_cost_per_token, input_cost_per_image_token, output_cost_per_image_token, input_cost_per_token_batches, output_cost_per_token_batches claude-fable-5: claude-fable-5-1: claude-haiku-4-5: claude-mythos-5: claude-mythos-5-1: claude-opus-4-5: claude-opus-4-6: claude-opus-4-7: claude-opus-4-8: claude-opus-5: claude-sonnet-4-5: claude-sonnet-4-6: claude-sonnet-5: davinci-002: input_cost_per_token_batches, output_cost_per_token_batches deep-research-pro-preview-12-2025: cache_read_input_token_cost together_ai/deepseek-ai/deepseek-coder-33b-instruct: input_cost_per_token, output_cost_per_token together_ai/deepseek-ai/DeepSeek-R1-0528:	2026-09-13 04:55:39 +00:00
joshua-berri	c1d0d29d01	fix(guardrails): import ModelResponse lazily to avoid cyclic import alert ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-13 04:50:29 +00:00
yucheng	e3fe26b43d	Merge remote-tracking branch 'origin/main' into litellm_agent365_mcp_guardrail	2026-09-13 04:47:13 +00:00
shivam	d1a4d678dc	Merge remote-tracking branch 'origin/litellm_bedrock_sanitize_tool_use_id' into litellm_bedrock_sanitize_tool_use_id	2026-09-13 04:24:46 +00:00
yucheng	c65ace9164	Merge remote-tracking branch 'origin/main' into litellm_model_armor_logging_only_streaming	2026-09-13 04:24:44 +00:00
yassin	eb48850a1c	feat(proxy): bind JWT claims to registered agents via agent_id_jwt_field ... JWT auth validated Entra app tokens but never carried an agent identity into the authenticated principal, so agent policies (trace id requirement, per-agent MCP restrictions, agent spend attribution) only applied to virtual keys bound to an agent. A new litellm_jwtauth field, agent_id_jwt_field, names the claim (dot notation supported) that is matched against a registered agent's id, then name; the canonical agent_id flows through the standard and proxy-admin JWT paths, and a configured claim naming no registered agent fails closed with 403 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-13 04:24:20 +00:00
shivam	a28e595a9d	Merge remote-tracking branch 'origin/main' into litellm_fix_realtime_cached_audio_cost	2026-09-13 04:24:18 +00:00
yassin	5d0a6e3a78	Merge remote-tracking branch 'origin/main' into litellm_flat_retry_records	2026-09-13 04:24:08 +00:00
shivam	e08f3cc021	Merge remote-tracking branch 'origin/main' into litellm_bedrock_sanitize_tool_use_id	2026-09-13 04:24:07 +00:00
yassin	268b9b167f	Merge remote-tracking branch 'origin/main' into litellm_proxy_error_headers_from_litellm_response_headers	2026-09-13 04:24:01 +00:00
yuneng-jiang	30f33a949b	Merge pull request #40943 from BerriAI/litellm_internal_staging ... chore(ci): remerge internal staging	2026-09-12 21:19:33 -07:00
Mateo Wang	c2c2a623c0	Merge pull request #39846 from BerriAI/litellm_bedrock_mantle_govcloud_cost_row ... fix(bedrock_mantle): price GovCloud regions from the regional cost row and accept region-prefixed model names	2026-09-12 21:13:58 -07:00
devin-ai-integration[bot]	62b3a93219	build(deps): bump smol-toml to 1.8.0 to clear GHSA-7w5x-hrqm-74c2 in osv-scan (#40478) ... Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-12 21:13:54 -07:00
Mateo Wang	b1a61f510c	Merge pull request #35918 from Lee-Si-Yoon/feat/friendli-model-metadata-sync ... feat(friendli): auto-sync Friendli model metadata into price registry	2026-09-12 21:13:52 -07:00
Shivam Rawat	e8d671c94a	Merge pull request #36585 from BerriAI/litellm_remove_user_soft_budget_docstring ... docs(user endpoints): remove unsupported soft_budget param from user docstrings	2026-09-12 21:13:46 -07:00
devin-ai-integration[bot]	8851148330	fix(router): preserve Azure Entra ID params in reusable credentials (#40889) ... CredentialLiteLLMParams omitted tenant_id, client_id, client_secret, azure_scope, azure_username and azure_password, so the strict dump used by credential reuse and Azure client init dropped them and the reused credential ended with no auth at all Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-12 21:13:45 -07:00
Yassin Kortam	036bfc08fc	docs(e2e): ban unit tests under tests/e2e (#33852) ... The e2e harness exists to prove product features end to end against a live proxy. The prior Hard Rule carved out an exception for "tests that cover the harness itself" and pointed at coverage_registry/test_collector.py, which in practice invited unit tests of harness helpers to be staged alongside e2e work. That is the wrong tool: harness logic that is worth locking down does not need a mock-driven unit test living under tests/e2e. Drop the carve-out. The Hard Rule now reads that no unit tests of any kind belong under tests/e2e, and the passing mention of unmarked harness coverage in the transport section is removed so the doc no longer contradicts itself. coverage_registry/test_collector.py still exists on disk and is left in place for now; whether to relocate or remove it is a separate decision.	2026-09-12 21:13:43 -07:00
Mateo Wang	939d320246	Merge pull request #40618 from BerriAI/litellm_pr_template_affected_release ... docs(github): add an Affected release section to the PR template	2026-09-12 21:13:38 -07:00
devin-ai-integration[bot]	77dc1a6c03	fix(anthropic-adapter): surface mid-stream provider errors as Anthropic error events (#33352) ... * fix(anthropic-adapter): surface mid-stream provider errors as Anthropic error events Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * style(anthropic-adapter): drop added comments per repo convention Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>	2026-09-12 21:13:35 -07:00
Mateo Wang	386d29ee67	Merge pull request #38867 from BerriAI/litellm_hide_admin_tabs_view_only ... fix(ui): hide admin write-form tabs on the models page from view-only admins	2026-09-12 21:13:34 -07:00
ryan-crabbe-berri	760119681c	Merge pull request #40814 from BerriAI/litellm_gate_health_services_alert_tests ... fix(proxy): gate the webhook test alert on proxy admins	2026-09-12 21:13:30 -07:00
Mateo Wang	70e3f5a02e	Merge pull request #39836 from BerriAI/litellm_lit_6975_bedrock_files_delete_list ... feat(bedrock): support file delete and list for S3-backed managed files	2026-09-12 21:13:27 -07:00
ryan-crabbe-berri	1ce3690257	Merge pull request #40657 from BerriAI/litellm_lit_7358_session_token_grant_resolver ... fix(auth): refresh lite login session token grants from the live user and team rows	2026-09-12 21:13:25 -07:00
Mateo Wang	a978ad2227	Merge pull request #39068 from BerriAI/litellm_spend_log_request_id_call_id ... fix(spend_logs): store litellm_call_id and match it in request_id lookups	2026-09-12 21:12:57 -07:00
yuneng-jiang	15789ae39e	Merge pull request #40942 from BerriAI/litellm_internal_staging ... chore(ci): promote internal staging to main	2026-09-12 21:11:12 -07:00
yuneng-jiang	daa2b0248a	Merge pull request #40172 from BerriAI/litellm_remove_main_guard ... ci: remove main branch source guard	2026-09-12 21:10:34 -07:00
mateo-berri	8608a03bd8	Merge origin/litellm_internal_staging into litellm_spend_log_request_id_call_id ... Keeps the base's rule that a non-admin id lookup matching no spend-log row answers 403, so the detail route never consults cold storage without an owner row	2026-09-12 21:04:25 -07:00
kerry-berri	9ae727bc8e	Merge pull request #40929 from BerriAI/litellm_fireworks_short_key_lookup ... fix(fireworks): resolve short model names to long cost map keys	2026-09-12 20:49:44 -07:00
Devin AI	7a0ea9867d	docs(fireworks): drop provider example from get_model_cost_key docstring ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-13 03:38:14 +00:00
Devin AI	f1fe61af06	docs(fireworks): explain what get_model_cost_key is for ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-13 03:36:00 +00:00
mateo-berri	299cd084f9	refactor(prompt_templates): share the tool use id sanitizer between the Anthropic and Bedrock paths	2026-09-12 20:29:16 -07:00
Devin AI	a519d805bb	refactor(fireworks): resolve cost map key through a provider config hook ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-13 03:29:11 +00:00
mateo-berri	a635d7be6a	fix(guardrails): write per-message guardrail rewrites back onto Responses input items ... A guardrail that answers one rewritten text per message it saw no longer matches the texts the Responses handler extracted once the request carries instructions or tool items, so the rewrite was rejected with a 500. Spread such an answer over the structured messages' text slots and write it back through the structured path, have Prompt Security modify return structured_messages directly, and give the chat completions pairing the same named rejection instead of a silent misalignment when the counts differ.	2026-09-12 20:29:09 -07:00
Devin AI	0904051fda	refactor(fireworks): move cost map key construction under llms/ ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-13 03:24:23 +00:00
mateo-berri	74bc22c574	fix(logging): collapse constant-byte base64 payloads and keep only hex and decimal runs ... A run over MAX_BASE64_LENGTH_STDOUT_LOG now stays in the log line only when it is hex or decimal with at least two distinct characters. Collapsing only mixed-case runs let every constant-byte payload through: 0x00 encodes to AAAA, 0x01 to AQEB, 0x55 to VVVV, 0xAA to qqqq, so a zero-filled upload still paid the full secret regex. The two traceback tests that raised a 100,000-character run of one letter now raise the same text the other length-cap tests use, since a single-letter run is exactly the shape the collapse treats as a constant-byte payload	2026-09-12 20:02:36 -07:00
mateo-berri	0d47d6ca58	fix(logging): collapse base64 runs in tracebacks and leave single-case runs alone ... Only mixed-case runs of the base64 alphabet collapse now, so a long hex digest, numeric id, or padding run stays in the debug line. The truncation filter also formats the traceback at every level and collapses base64 runs in it before the secret regex sees it, instead of only capping its length at INFO and above	2026-09-12 19:42:45 -07:00
mateo-berri	5fcbf91730	fix(logging): scan each log record once and collapse base64 payloads before the secret regex ... Since #37391 every log record went through the secret-redaction regex twice, once in the filter and again in the formatter, and the formatter pass ran on the whole formatted line. At DEBUG level a multi-megabyte request body (a multi-page PDF upload to /v1/ocr) turned each of those lines into ten seconds of synchronous regex work on the event loop, long enough for a Kubernetes liveness probe to restart the pod mid-request. The filter is now the complete scrubber (message, exception text, stack info, and extras) and stamps the record, so the formatters skip records that are already clean. The stdout truncation filter also collapses base64 runs longer than MAX_BASE64_LENGTH_STDOUT_LOG (4096 by default) at every level before the secret regex sees them, so a debug line carrying a request body costs milliseconds instead of seconds.	2026-09-12 19:17:26 -07:00
Mateo Wang	e240997529	Merge pull request #40912 from BerriAI/litellm_logging_worker_timeout_summary ... fix(logging): log one bounded summary for a burst of timed-out LoggingWorker callbacks	2026-09-12 19:12:22 -07:00
joshua-berri	b7f4258e32	Merge pull request #40923 from BerriAI/litellm_fix_mcp_delegated_auth_tracking_6970 ... fix(mcp): require admission for delegated OAuth	2026-09-12 19:11:35 -07:00
devin-ai-integration[bot]	56045503db	fix(proxy): bound tool and guardrail index create_many by the spend-log statement budgets (#40561) ... * fix(proxy): bound tool and guardrail index create_many by the spend-log statement budgets One flush drains up to MAX_LOGS_PER_INTERVAL source transactions or logs, but a transaction fans out to one LiteLLM_SpendLogToolIndex row per tool and a log to one LiteLLM_SpendLogGuardrailIndex row per guardrail, so the index create_many payload was unbounded. Both index writes now go through spend_log_write_batches(SPEND_LOG_WRITE_BATCH_MAX_BYTES, SPEND_LOG_WRITE_BATCH_MAX_ROWS). The tool index write moves out of the rollup batch_() so the split reduces the query-engine payload; replayed index rows are no-ops under skip_duplicates, and the daily rollup upserts stay in one transaction Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): pin the row budget in the index fan-out tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-12 19:00:47 -07:00
ryan-crabbe-berri	67dd150fed	Merge pull request #36363 from joshgarnett/bugfix/responses-reasoning-object-to-effort ... fix(responses): translate the reasoning object into a chat-completion reasoning effort	2026-09-12 18:52:25 -07:00
yuneng-jiang	16f6d92453	Merge pull request #40933 from BerriAI/litellm_internal_staging ... chore(ci): promote internal staging to main	2026-09-12 18:51:00 -07:00