yassin
e41b3bd13f
test(router): annotate return types of team cooldown test helpers
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 09:50:12 +00:00
yassin
e61b6bfd5f
fix(router): classify pass-through cooldown against pass-through deployments only
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 09:48:35 +00:00
mateo-berri
6264bd84bf
chore(ui): regenerate dashboard API types
2026-09-13 02:43:36 -07:00
mateo-berri
dd173a0b1b
fix(guardrails): count the PANW latest-user scan over the hoisted system prompt
2026-09-13 02:43:35 -07:00
yassin
9080f0904a
fix(router): ignore blocked siblings when checking team model cooldown alternatives
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 09:35:54 +00:00
mateo-berri
c246372859
fix(responses): import BaseLLMException lazily and collect stream chunks via anext
...
Move the BaseLLMException import into _map_error_event_exception so the
module no longer imports it at load time, clearing the module-level cyclic
import CodeQL flagged. The class is used only on the cold error path.
Replace the mutable list-append test collector with aiter/anext so the
regression tests read the stream immutably.
2026-09-13 02:35:53 -07:00
yassin
10f411e60d
fix(router): name the all-deployments-in-cooldown error on 429 responses
...
RouterRateLimitError now carries the model group's deployment ids so it
can tell when every deployment is cooled down, and exposes that as
type=all_deployments_in_cooldown with an explicit message. A partial
cooldown keeps type=rate_limit_error. Either way the proxy no longer
reports type=internal_server_error next to code 429
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 09:34:16 +00:00
mateo-berri
37447c98f7
fix(guardrails): reject per-message texts that cannot land on a string input or a Messages request
2026-09-13 02:28:43 -07:00
yassin
d0a846c8be
test(router): cover team_model_has_alternatives directly in the mapped router test
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 09:27:33 +00:00
mateo-berri
825e4f17e9
fix(sdk): carry body and proxy headers on relayed litellm errors and content policy blocks too
2026-09-13 02:23:34 -07:00
yassin
330ba7cbf9
fix(ui): show the team alias on the model info page and in its raw JSON
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 09:21:09 +00:00
yucheng
685c654298
refactor(proxy): assert separate login counter stores in the spray regression test instead of a comment
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 09:18:39 +00:00
mateo-berri
f7e9277032
fix(guardrails): validate rewritten tool_use arguments with a typed adapter
2026-09-13 02:13:17 -07:00
mateo-berri
e732a484f6
fix(responses): hoist Codex additional_tools input items into the chat bridge tools
2026-09-13 02:11:59 -07:00
mateo-berri
073d4fe2b0
fix(responses): route mid-stream error events through exception_type so content_policy_fallbacks fire
...
Mid-stream error events on the streaming Responses API were all raised as
APIError, so a content_policy_violation event never matched the router's
content-policy fallback dispatch and the client got the raw error instead
of the fallback model's answer. Map each error event's code and status
through the existing exception_type mapping, matching the non-streaming
path, and unwrap the typed ContentPolicyViolationError and
ContextWindowExceededError so the router routes them to the configured
content_policy_fallbacks and context_window_fallbacks.
2026-09-13 02:11:06 -07:00
yassin
76b26e41ab
fix(router): cool down team deployments on 429 when a sibling serves the same public model
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 09:07:53 +00:00
mateo-berri
c0c0c9a9eb
fix(sdk): keep body and proxy headers on BadRequestError mapped from a litellm_proxy 400
...
The generic 400 branch of the OpenAI exception mapper dropped the wire body and no
branch carried the response headers, so an application calling a LiteLLM proxy through
a litellm_proxy/ model could not tell a guardrail block from any other failure without
walking __cause__. BadRequestError now takes headers, filled for a litellm_proxy
upstream, and the generic branch passes the body. The proxy edge treats the literal
"None" type and param an older proxy sends as absent and stops forwarding an upstream
proxy's date and server headers.
2026-09-13 02:05:10 -07:00
yucheng
05fe17e027
test(proxy): drop the section banner comment from the login tests
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 09:02:45 +00:00
yucheng
ce82033f62
refactor(proxy): inject settings and Redis cache into LoginThrottle.from_request
...
Removes the runtime import of proxy_server from login_throttle so the
throttle module no longer participates in the import cycle CodeQL
flagged (py/cyclic-import). Callers pass general_settings and
redis_usage_cache explicitly; behavior is unchanged.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 09:00:20 +00:00
mateo-berri
6f7882db34
fix(types): import CachedTokensDetails on its own line in openai.py
...
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
CodeQL resolves `from openai import Omit` in litellm/types/llms/openai.py to the
module itself, so every importer of a name whose definition line is in the diff
is reported as an unsafe cyclic import. 76ae35dfcd
2026-09-13 01:54:12 -07:00
joshua-berri
49b2d71057
test(guardrails): type the native lifecycle logging_only test double
...
ai-gateway image / ai-gateway release image (push) Waiting to run
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 08:47:47 +00:00
mateo-berri
20d80b5420
fix(guardrails): scan each choice's tool-call arguments apart on n>1 streams and log why a rewrite was discarded
...
The rebuilt streamed response keyed tool-call fragments by tool index alone, so
on n>1 chat streams the two choices' argument fragments were concatenated into
one string and post_call guardrails scanned garbled JSON. Fragments are now
keyed by (choice index, tool index).
When a guardrail's rewrite cannot be written back to the stream (multi-choice
streams, a rewrite that adds or drops a tool call, legacy-hook shapes the
translation cannot rescan), the pipeline now logs a warning naming the
guardrail and the exact reason before releasing the original stream.
Also commits the regenerated dashboard API types that make check produced.
2026-09-13 01:45:07 -07:00
mateo-berri
cccff657cf
fix(guardrails): scan the Anthropic top-level system prompt and tool_use arguments
2026-09-13 01:35:19 -07:00
mateo-berri
31bd4d34ed
fix(rerank): stamp a fresh response id when Voyage, watsonx, or Fireworks omit one
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
2026-09-13 01:34:22 -07:00
yucheng
9ca6307b9d
chore(ui): regenerate schema.d.ts after merging main
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 08:28:06 +00:00
yucheng
853a39442a
Merge remote-tracking branch 'origin/main' into litellm_lit5285_login_rate_limit_v2
2026-09-13 08:27:23 +00:00
mateo-berri
23a98cb851
chore(ui): regenerate schema.d.ts after merging main
2026-09-13 01:22:49 -07:00
mateo-berri
1760fe6284
refactor(guardrails): return a fresh inputs mapping from the Prompt Security modify branch
2026-09-13 01:22:49 -07:00
mateo-berri
93f3911b32
fix(guardrails): drop the types import CodeQL reads as a package cycle
2026-09-13 01:22:49 -07:00
mateo-berri
76ae35dfcd
fix(types): break the CachedTokensDetails import cycle
...
CodeQL flagged two module-level cyclic imports introduced by defining
CachedTokensDetails in litellm.types.llms.openai and importing it from
litellm.types.utils and litellm.cost_calculator. The class now lives in
litellm.types.llms.base, which imports nothing from litellm, and every
user imports it from there.
Also pins that combining realtime usages where only one response.done
carries cached_tokens_details keeps the earlier modality split in both
orders, and commits the regenerated dashboard API types.
2026-09-13 01:14:35 -07:00
mateo-berri
d26e007229
Merge remote-tracking branch 'origin/main' into litellm_responses_per_message_guardrail_rewrite
2026-09-13 01:05:36 -07:00
mateo-berri
09314f239c
fix(guardrails): hand per-message rewrites back as structured_messages
...
A guardrail that rewrites text per chat message now returns the rewritten
rows as structured_messages instead of only texts, so the Responses and
chat handlers write the rewrite back through the structured path. The
generic guardrail API response accepts an optional structured_messages
list, Prompt Security modify builds one from modified_messages, and rows a
server echoes back exactly as shown are restored to the original row
objects because the request model drops undeclared keys. Texts-only
per-message answers keep the named rejection on both endpoints.
2026-09-13 00:54:29 -07:00
mateo-berri
4d2352d0b5
fix(cost): bill per-query priced rerank deployments from their router model id
2026-09-13 00:45:33 -07:00
yucheng
a2501d3b59
chore: re-run review bots on the reopened PR tip
...
ai-gateway image / ai-gateway release image (push) Waiting to run
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 07:06:56 +00:00
mateo-berri
af363fc29c
Merge remote-tracking branch 'origin/main' into litellm_lit4995_vertex_rerank_search_units
2026-09-13 00:03:04 -07:00
mateo-berri
4c022a3089
feat(pricing): add azure gpt-chat-latest global and data zone rates
2026-09-12 23:34:12 -07:00
yucheng
d7623112fa
fix(mcp): hand Agent 365 the admin description override and clear the LIT001 and schema gates
...
get_listed_tool now applies tool_name_to_description so the guardrail evaluates the description clients saw, with a regression test. The listed-tools cache and tool_input_schema fields use Mapping annotations so the change adds no LIT001 findings, and schema.d.ts is regenerated with npm run gen:api
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 05:15:33 +00:00
yassin
e0cf63c5fa
Merge remote-tracking branch 'origin/main' into litellm_per_issuer_virtual_key_claim_field
2026-09-13 05:00:54 +00:00
berriai-litellm-provider-info-sync[bot]
6423acc11a
chore(prices): sync prices for 5 providers: 278 models, 34 new
...
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
fireworks_ai/accounts/fireworks/models/deepseek-v4-flash-0731: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/deepseek-v4-flash-0731: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/accounts/fireworks/models/deepseek-v4-flash-vision-exp:
fireworks_ai/deepseek-v4-flash-vision-exp:
fireworks_ai/accounts/fireworks/models/deepseek-v4-pro: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost, input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/deepseek-v4-pro: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost, input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/accounts/fireworks/models/deepseek-v4-pro-0813: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/accounts/fireworks/models/deepseek-v4p1-flash: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/deepseek-v4p1-flash: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/accounts/fireworks/models/glm-5p2: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/glm-5p2: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/accounts/fireworks/models/glm-5p3: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/accounts/fireworks/models/glm-5p3-flash: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/accounts/fireworks/models/gpt-oss-120b: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/gpt-oss-120b: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/accounts/fireworks/models/kimi-k2p6: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/kimi-k2p6: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/accounts/fireworks/models/kimi-k2p7-code: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/kimi-k2p7-code: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/accounts/fireworks/models/kimi-k3: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/kimi-k3: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/accounts/fireworks/models/minimax-m2p7: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/minimax-m2p7: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/accounts/fireworks/models/minimax-m3: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/minimax-m3: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/accounts/fireworks/models/muse-glimmer-30b:
fireworks_ai/muse-glimmer-30b:
fireworks_ai/accounts/fireworks/models/nemotron-3-ultra-nvfp4:
fireworks_ai/nemotron-3-ultra-nvfp4:
fireworks_ai/accounts/fireworks/models/qwen3-embedding-8b:
fireworks_ai/accounts/fireworks/models/qwen3-reranker-8b: input_cost_per_token
fireworks_ai/accounts/fireworks/models/qwen3p7-plus:
fireworks_ai/qwen3p7-plus:
fireworks_ai/accounts/fireworks/models/qwen3p8-max: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/qwen3p8-max: input_cost_per_token_priority, output_cost_per_token_priority, cache_read_input_token_cost_priority
fireworks_ai/accounts/fireworks/routers/glm-5p2-fast:
fireworks_ai/accounts/fireworks/routers/glm-5p3-fast: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
fireworks_ai/accounts/fireworks/routers/kimi-k3-fast:
together_ai/arcee-ai/trinity-mini: input_cost_per_token, output_cost_per_token
together_ai/arize-ai/qwen-2-1.5b-instruct:
babbage-002: input_cost_per_token_batches, output_cost_per_token_batches
chat-latest:
chatgpt-image-latest: output_cost_per_token, input_cost_per_image_token, output_cost_per_image_token, input_cost_per_token_batches, output_cost_per_token_batches
claude-fable-5:
claude-fable-5-1:
claude-haiku-4-5:
claude-mythos-5:
claude-mythos-5-1:
claude-opus-4-5:
claude-opus-4-6:
claude-opus-4-7:
claude-opus-4-8:
claude-opus-5:
claude-sonnet-4-5:
claude-sonnet-4-6:
claude-sonnet-5:
davinci-002: input_cost_per_token_batches, output_cost_per_token_batches
deep-research-pro-preview-12-2025: cache_read_input_token_cost
together_ai/deepseek-ai/deepseek-coder-33b-instruct: input_cost_per_token, output_cost_per_token
together_ai/deepseek-ai/DeepSeek-R1-0528:
2026-09-13 04:55:39 +00:00
joshua-berri
c1d0d29d01
fix(guardrails): import ModelResponse lazily to avoid cyclic import alert
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 04:50:29 +00:00
yucheng
e3fe26b43d
Merge remote-tracking branch 'origin/main' into litellm_agent365_mcp_guardrail
2026-09-13 04:47:13 +00:00
yucheng
c3a7b7c3ee
fix(proxy): warn about per-worker login counters even without general_settings
...
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 04:36:41 +00:00
shivam
d1a4d678dc
Merge remote-tracking branch 'origin/litellm_bedrock_sanitize_tool_use_id' into litellm_bedrock_sanitize_tool_use_id
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
2026-09-13 04:24:46 +00:00
yucheng
c65ace9164
Merge remote-tracking branch 'origin/main' into litellm_model_armor_logging_only_streaming
2026-09-13 04:24:44 +00:00
yassin
eb48850a1c
feat(proxy): bind JWT claims to registered agents via agent_id_jwt_field
...
ai-gateway image / ai-gateway release image (push) Waiting to run
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
JWT auth validated Entra app tokens but never carried an agent identity
into the authenticated principal, so agent policies (trace id
requirement, per-agent MCP restrictions, agent spend attribution) only
applied to virtual keys bound to an agent. A new litellm_jwtauth field,
agent_id_jwt_field, names the claim (dot notation supported) that is
matched against a registered agent's id, then name; the canonical
agent_id flows through the standard and proxy-admin JWT paths, and a
configured claim naming no registered agent fails closed with 403
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-13 04:24:20 +00:00
shivam
a28e595a9d
Merge remote-tracking branch 'origin/main' into litellm_fix_realtime_cached_audio_cost
2026-09-13 04:24:18 +00:00
yassin
5d0a6e3a78
Merge remote-tracking branch 'origin/main' into litellm_flat_retry_records
2026-09-13 04:24:08 +00:00
shivam
e08f3cc021
Merge remote-tracking branch 'origin/main' into litellm_bedrock_sanitize_tool_use_id
2026-09-13 04:24:07 +00:00
yassin
268b9b167f
Merge remote-tracking branch 'origin/main' into litellm_proxy_error_headers_from_litellm_response_headers
2026-09-13 04:24:01 +00:00
yuneng-jiang
30f33a949b
Merge pull request #40943 from BerriAI/litellm_internal_staging
...
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests / caching-local (push) Waiting to run
Unit Tests / core-utils (push) Waiting to run
Unit Tests / enterprise-package (push) Waiting to run
Unit Tests / enterprise-routing (push) Waiting to run
Unit Tests / integrations (push) Waiting to run
Unit Tests / All Other Providers (push) Waiting to run
Unit Tests / Vertex AI (push) Waiting to run
Unit Tests / misc (push) Waiting to run
Unit Tests / proxy-auth (push) Waiting to run
Unit Tests / proxy-endpoints (push) Waiting to run
Unit Tests / proxy-extras (push) Waiting to run
Unit Tests / proxy-infra (push) Waiting to run
Unit Tests / proxy-server (push) Waiting to run
Unit Tests / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
chore(ci): remerge internal staging
2026-09-12 21:19:33 -07:00
