The gemini-3.1-flash-image-preview model introduced a new pricing field
that was missing from the test's validation schema and cost_fields list.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Chocolatey v2.5.1 introduced interactive prompts that block CI. Add
--no-progress, --force flags and CHOCOLATEY_CONFIRM_ALL env var to
fully suppress user input in non-interactive environments.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(agentcore): handle JSON responses from agents using sync return
BedrockAgentCoreApp agents that use synchronous `return` (instead of
async `yield`) respond with Content-Type: application/json instead of
text/event-stream. The streaming parser only handles SSE format, silently
discarding the JSON body and returning empty content to the client.
This adds Content-Type detection in both sync and async streaming
wrappers — when application/json is received, the response is parsed
and converted to a single-chunk stream. Also extends _parse_json_response
with a fallback chain supporting multiple agent response schemas (standard
AgentCore, Strands framework, plain string, raw JSON fallback).
* fix(agentcore): add dict-type guard to _parse_json_response
Prevent AttributeError when json.loads() returns a non-dict
(e.g. JSON array or primitive) by adding an isinstance check
at the top of _parse_json_response. Non-dict values fall back
to raw JSON string content.
* fix(agentcore): handle malformed JSON and split streaming chunks
- Wrap json.loads() in try/except in both sync and async streaming
wrappers so malformed JSON bodies raise a structured BedrockError
instead of a raw JSONDecodeError
- Split the JSON-fallback streaming path into two chunks (content
chunk with finish_reason=None, then stop sentinel with empty delta)
to match the SSE path convention
* fix(agentcore): catch IO errors in streaming JSON path + async error test
- Broaden except clause to catch both json.JSONDecodeError and IO-level
exceptions (httpx.ReadError, etc.) from response.read()/aread(), so
all failures surface as structured BedrockError
- Add async malformed-JSON test to mirror the sync test coverage
* feat(proxy): add Prisma DB pool and engine health metrics to Prometheus
Add a PrismaMetricsCollector that periodically queries pg_stat_activity
and the Prisma engine process to expose connection pool and engine health
as Prometheus gauges/counters. Auto-enabled when prometheus_system is in
service_callback.
New metrics:
- litellm_db_pool_active_connections (Gauge)
- litellm_db_pool_idle_connections (Gauge)
- litellm_db_pool_total_connections (Gauge)
- litellm_db_pool_waiting_connections (Gauge)
- litellm_db_engine_up (Gauge)
- litellm_db_engine_restarts_total (Counter)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address Greptile review feedback
- Only increment engine_restarts counter on heavy reconnects (engine
actually dead), not lightweight network-blip reconnects
- Fix potential KeyError in _get_or_create_gauge/counter fallback path
when REGISTRY._names_to_collectors is absent
- Rename litellm_db_pool_waiting_connections to
litellm_db_pool_lock_waiting_connections to clarify it measures lock
contention, not pool slot queuing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: warn when prometheus_system enabled but watchdog disabled
Log a warning when users have prometheus_system in service_callback
but PRISMA_HEALTH_WATCHDOG_ENABLED=false, since DB pool and engine
metrics won't be collected in that configuration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* ci: retrigger CI checks
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: use labeled gauge for DB pool connection metrics
Replace 3 separate pool gauges (active, idle, total) with a single
`litellm_db_pool_connections` gauge using a `state` label. This is more
Prometheus-idiomatic and exposes all pg_stat_activity states (active,
idle, idle in transaction, etc.) without ambiguity about what "total"
includes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address Greptile review — stale labels and fallback re-registration
- Zero out known pg_stat_activity states that are absent from the current
query result, preventing stale gauge values from persisting.
- Simplify _get_or_create_gauge/counter by removing the fallback loop
that could re-register an already-registered metric (ValueError).
- Add test for stale label clearing across collection cycles.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: include "unknown" in _PG_STATES for stale label clearing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: collect immediately on start and consolidate into single query
- Move sleep to end of loop so metrics appear on /metrics immediately
after startup instead of after a 30s delay.
- Combine pool state and lock waiting queries into a single SQL query
using conditional aggregation, halving per-cycle DB overhead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent tight spin loop on collection error
Move asyncio.sleep outside the try/except so it always executes even
when _collect_engine_health() or _collect_pool_metrics() raises.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add multiprocess_mode to _get_or_create_gauge initialization
- Include `multiprocess_mode` parameter to properly support multiprocessing in Gauge creation.
- Ensure consistent behavior for labeled and unlabeled Gauges.
* fix: handle invalid env var and document watchdog prerequisite
- Add try/except ValueError for PRISMA_METRICS_COLLECTION_INTERVAL_SECONDS
to prevent proxy startup crash on non-numeric values (e.g. "30s")
- Document that DB metrics require both prometheus_system callback and
PRISMA_HEALTH_WATCHDOG_ENABLED=true
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use defensive null coalescing for query_raw row values
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add invalid env var fallback test and fix mock signature
- Add test for non-numeric PRISMA_METRICS_COLLECTION_INTERVAL_SECONDS
- Add **kwargs to mock _patched_get_or_create_gauge for forward compat
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
When both enable_jwt_auth and enable_oauth2_auth are True, the proxy now
routes tokens based on their format:
- JWT tokens (3 dot-separated parts) -> JWT auth handler
- Opaque tokens -> OAuth2 auth handler
This enables using JWT for human users and OAuth2 for M2M (machine) clients
on the same LiteLLM instance. Previously, enabling OAuth2 would intercept
all tokens on LLM API routes before JWT auth could run.
When only one auth method is enabled, behavior is unchanged (backward compatible).
- Add _get_effort_level() to extract effective effort from string or dict
- Use effective_effort for xhigh validation, tool-drop, sampling, temperature guards
- Preserve dict format when it has summary/generate_summary for Responses API
- Add tests: xhigh-dict validation, none-dict for tools/sampling/temperature
- Update tests: dict-with-summary now preserved (not normalized)
Made-with: Cursor
When reasoning_effort is passed as a dict with additional fields like 'summary' or 'generate_summary', preserve the full dict format instead of normalizing it to a string. This ensures that when requests are routed to the OpenAI Responses API, all reasoning parameters are correctly included.
The normalization to string format now only happens for simple dicts with just the 'effort' key, which is appropriate for the Chat Completions API.
Fixes issue where summary field was being dropped when routing gpt-5.4+ requests with tools + reasoning to Responses API.
Made-with: Cursor
gpt-5.4-pro and gpt-5.4-pro-2026-03-05 do not support the
/v1/chat/completions endpoint — OpenAI returns a 404 with
"This is not a chat model". These models are responses-only,
like o3-pro and o1-pro.
Changes:
- Set mode from "chat" to "responses" for both model entries
- Update supported_endpoints to ["/v1/responses", "/v1/batch"]
- Add regression test for responses API bridge routing
FixesBerriAI/litellm#23014
PR #20950 added extra_body forwarding to Vertex AI Gemini. LiteLLM-internal
keys (cache, tags) were being merged into the request body, causing Vertex AI
to reject with 400: 'Unknown name "cache": Cannot find field.'
- Add _LITELLM_INTERNAL_EXTRA_BODY_KEYS frozenset (cache, tags)
- Skip these keys in _pop_and_merge_extra_body before merging
- Add regression tests for cache and tags stripping
Fixes regression from 1.79.3 → 1.81.12 when using proxy cache with
extra_body={"cache": {"use-cache": True, "ttl": 86400}}
Made-with: Cursor
Addresses Greptile review:
- dict(_input_schema) before mutation prevents cross-provider state leakage
- Test asserts original tool parameters dict is unchanged after call
Anthropic requires exactly one tool_result per tool_use. When
conversation history (e.g. from session resume/checkpoint restore)
contains duplicate tool result messages with the same tool_call_id,
the API rejects with: 'each tool_use must have a single result.
Found multiple tool_result blocks with id: <id>'.
This is already handled for Bedrock via _deduplicate_bedrock_tool_content()
but was missing from the Anthropic direct and Vertex AI partner paths,
which share sanitize_messages_for_tool_calling().
Fix: Add Case D to sanitize_messages_for_tool_calling() — after the
existing orphan detection passes, scan for duplicate tool_call_ids
and keep only the last occurrence (most complete result).
Added 3 unit tests: dedup with duplicates, no-op with unique IDs,
and behavior when modify_params=False.
Related issues: #11804, #11029, #6836, #1782, #151
Anthropic's API requires all tool input_schema to have type:'object'
at the root level. When OpenAI-format tools have parameters with a
missing or non-'object' type field (common with MCP tool servers),
the schema was passed through unchanged, causing Anthropic to reject
with: 'tools.N.custom.input_schema.type: Input should be object'.
The existing default handles the case where parameters is entirely
missing, but does not normalize schemas that ARE provided with a
wrong or absent type field.
Fix: After extracting _input_schema in _map_tool_helper(), ensure
type is set to 'object' and properties exists. This matches the
normalization already done implicitly by the Bedrock handler.
Added 4 unit tests covering: missing type, wrong type, valid schema
(no-op), and entirely missing parameters.
Related issues: #12020, #64, #1671
AgentsPanel no longer renders AgentCardGrid — it uses a Table directly.
The two tests that looked for agent-card-grid are updated to instead verify
the Actions column header appears for admins and is absent for non-admins.
