Claude's API returns assistant messages with empty text blocks
({"type": "text", "text": ""}) alongside tool_use blocks during
multi-turn tool-use conversations. These blocks are rejected when
sent back to the API with "text content blocks must be non-empty".
Sanitization already exists for other code paths (/v1/chat/completions
for both Anthropic and Bedrock), but NOT for the /v1/messages native
path. This adds the same treatment by stripping empty text blocks
from messages in async_anthropic_messages_handler before they are
forwarded to the provider.
Fixes#22930
* add new azure gpt models
* add versionless azure/gpt-5.4 models
* Undated azure/gpt-5.4 alias missing supports_service_tier
* indicate service tier support for azure/gpt-5.3-chat
* fix priority tier pricing for new azure/gpt models
The bug occurred when user data inadvertently contained reserved Python
keywords like 'self', 'params', or '__class__' as keys. When such a dict
was unpacked via **kwargs to LiteLLM_Params() or GenericLiteLLMParams(),
Python raised TypeError because 'self' was passed both implicitly and
as a keyword argument.
The fix:
- Add a Pydantic model_validator(mode='before') to GenericLiteLLMParams
that filters out reserved keys ('self', 'params', '__class__') before
validation
- Move the max_retries str-to-int conversion into the same validator
- Remove the custom __init__ methods from both GenericLiteLLMParams and
LiteLLM_Params, since the validator now handles the preprocessing
- Clean up unused VERTEX_CREDENTIALS_TYPES import
This fix applies to all classes that inherit from GenericLiteLLMParams,
including LiteLLM_Params and updateLiteLLMParams.
Added comprehensive tests in tests/test_litellm/test_litellm_params_reserved_keys.py
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
The test calls OpenAI's gpt-4o-audio-preview model which sometimes
doesn't return usage data in the streaming response. Fixed by:
- Adding @pytest.mark.flaky(retries=5, delay=2) for retry handling
- Fixing usage_obj loop to check chunk.usage is not None
- Skipping gracefully when OpenAI doesn't return usage data
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test_create_skill test was consistently failing in CI with a 500 from
Anthropic because the SKILL.md frontmatter always used the same hardcoded
name (test-skill-litellm). Since test_delete_skill is permanently skipped,
skills accumulate in the CI account, and re-creating with a duplicate name
triggers an Internal Server Error on Anthropic's side.
Fix: pass a timestamp-based unique_suffix to create_skill_zip so each run
produces a distinct skill name in the zip's SKILL.md frontmatter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When selecting a popular API from the quick-picker, show the 8 most
useful MCP tools for that API in a collapsible preview card. First 4
are shown by default; clicking "expand" shows all 8 with descriptions
on hover.
- Added `key_tools` array (8 tools each) to all 10 APIs in openapi_registry.json
- New `KeyToolsPreview` component in OpenAPIFormSection with expand/collapse
- Extended `OpenAPIRegistryEntry` type with `key_tools?: OpenAPIKeyTool[]`
- New `openapi_registry.json` with 10 well-known APIs (GitHub, Atlassian, Figma, Google, Stripe, HubSpot, Notion, Slack, Shopify, Snowflake) — each with validated spec URLs and OAuth 2.0 endpoints
- Backend endpoint `GET /v1/mcp/registry.json` to serve the registry (reads fresh from disk)
- `OpenAPIQuickPicker` component: logo grid for popular APIs with letter fallback for broken images
- `OpenAPIFormSection` component: encapsulates picker + spec URL input as a clean unit
- When selecting a preset, spec URL and OAuth fields are pre-filled automatically
- Fixed `useTestMCPConnection`: for OpenAPI transport, tools load from the spec as soon as the URL is set — no auth type or OAuth token required
- Validated all spec URLs are reachable; removed Linear (GraphQL-only, no REST spec)
The Claude Agent SDK sends max_tokens=32000 for unrecognized model names
(like "bedrock-nova-pro"), which exceeds Nova Pro's 10,000 limit. Enable
modify_params in the test proxy config so LiteLLM clamps max_tokens to the
model's actual limit. Also swap nova-premier to nova-pro since premier
requires provisioned throughput unavailable in CI.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test was flaky under pytest-xdist parallel execution because it used
async acompletion (which runs completion() in a thread pool via
run_in_executor) and relied on shared global state (known_tokenizer_config,
iam_token_cache, module_level_client) that could be modified by other tests
running in parallel. Failures were silently swallowed by a broad try/except,
causing mock_post.call_count to remain 0.
Fix:
- Convert from async acompletion to sync completion, matching every other
test in the file. The test's intent is verifying prompt transformation,
not async behavior.
- Use monkeypatch.setitem for known_tokenizer_config to ensure proper
teardown isolation.
- Remove unnecessary mock layers (async template fetchers, iam_token_cache
pre-population, mock completion response) that were only needed for the
async code path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When retrieving a batch via the unified batch ID path, only unified_batch_id
was set on _hidden_params but model_id was missing. The managed files hook
requires both to encode output_file_id into a managed ID.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The SearXNG search tests were failing in CI because they depend on a live
SearXNG instance that returns results. Since this provider is used by a
very small subset of customers, replace the flaky integration tests with
deterministic unit tests that validate request payloads, URL construction,
response parsing, and header configuration without requiring external infra.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The gemini-3.1-flash-image-preview model introduced a new pricing field
that was missing from the test's validation schema and cost_fields list.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Chocolatey v2.5.1 introduced interactive prompts that block CI. Add
--no-progress, --force flags and CHOCOLATEY_CONFIRM_ALL env var to
fully suppress user input in non-interactive environments.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(agentcore): handle JSON responses from agents using sync return
BedrockAgentCoreApp agents that use synchronous `return` (instead of
async `yield`) respond with Content-Type: application/json instead of
text/event-stream. The streaming parser only handles SSE format, silently
discarding the JSON body and returning empty content to the client.
This adds Content-Type detection in both sync and async streaming
wrappers — when application/json is received, the response is parsed
and converted to a single-chunk stream. Also extends _parse_json_response
with a fallback chain supporting multiple agent response schemas (standard
AgentCore, Strands framework, plain string, raw JSON fallback).
* fix(agentcore): add dict-type guard to _parse_json_response
Prevent AttributeError when json.loads() returns a non-dict
(e.g. JSON array or primitive) by adding an isinstance check
at the top of _parse_json_response. Non-dict values fall back
to raw JSON string content.
* fix(agentcore): handle malformed JSON and split streaming chunks
- Wrap json.loads() in try/except in both sync and async streaming
wrappers so malformed JSON bodies raise a structured BedrockError
instead of a raw JSONDecodeError
- Split the JSON-fallback streaming path into two chunks (content
chunk with finish_reason=None, then stop sentinel with empty delta)
to match the SSE path convention
* fix(agentcore): catch IO errors in streaming JSON path + async error test
- Broaden except clause to catch both json.JSONDecodeError and IO-level
exceptions (httpx.ReadError, etc.) from response.read()/aread(), so
all failures surface as structured BedrockError
- Add async malformed-JSON test to mirror the sync test coverage
* feat(proxy): add Prisma DB pool and engine health metrics to Prometheus
Add a PrismaMetricsCollector that periodically queries pg_stat_activity
and the Prisma engine process to expose connection pool and engine health
as Prometheus gauges/counters. Auto-enabled when prometheus_system is in
service_callback.
New metrics:
- litellm_db_pool_active_connections (Gauge)
- litellm_db_pool_idle_connections (Gauge)
- litellm_db_pool_total_connections (Gauge)
- litellm_db_pool_waiting_connections (Gauge)
- litellm_db_engine_up (Gauge)
- litellm_db_engine_restarts_total (Counter)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address Greptile review feedback
- Only increment engine_restarts counter on heavy reconnects (engine
actually dead), not lightweight network-blip reconnects
- Fix potential KeyError in _get_or_create_gauge/counter fallback path
when REGISTRY._names_to_collectors is absent
- Rename litellm_db_pool_waiting_connections to
litellm_db_pool_lock_waiting_connections to clarify it measures lock
contention, not pool slot queuing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: warn when prometheus_system enabled but watchdog disabled
Log a warning when users have prometheus_system in service_callback
but PRISMA_HEALTH_WATCHDOG_ENABLED=false, since DB pool and engine
metrics won't be collected in that configuration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* ci: retrigger CI checks
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: use labeled gauge for DB pool connection metrics
Replace 3 separate pool gauges (active, idle, total) with a single
`litellm_db_pool_connections` gauge using a `state` label. This is more
Prometheus-idiomatic and exposes all pg_stat_activity states (active,
idle, idle in transaction, etc.) without ambiguity about what "total"
includes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address Greptile review — stale labels and fallback re-registration
- Zero out known pg_stat_activity states that are absent from the current
query result, preventing stale gauge values from persisting.
- Simplify _get_or_create_gauge/counter by removing the fallback loop
that could re-register an already-registered metric (ValueError).
- Add test for stale label clearing across collection cycles.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: include "unknown" in _PG_STATES for stale label clearing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: collect immediately on start and consolidate into single query
- Move sleep to end of loop so metrics appear on /metrics immediately
after startup instead of after a 30s delay.
- Combine pool state and lock waiting queries into a single SQL query
using conditional aggregation, halving per-cycle DB overhead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent tight spin loop on collection error
Move asyncio.sleep outside the try/except so it always executes even
when _collect_engine_health() or _collect_pool_metrics() raises.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add multiprocess_mode to _get_or_create_gauge initialization
- Include `multiprocess_mode` parameter to properly support multiprocessing in Gauge creation.
- Ensure consistent behavior for labeled and unlabeled Gauges.
* fix: handle invalid env var and document watchdog prerequisite
- Add try/except ValueError for PRISMA_METRICS_COLLECTION_INTERVAL_SECONDS
to prevent proxy startup crash on non-numeric values (e.g. "30s")
- Document that DB metrics require both prometheus_system callback and
PRISMA_HEALTH_WATCHDOG_ENABLED=true
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use defensive null coalescing for query_raw row values
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add invalid env var fallback test and fix mock signature
- Add test for non-numeric PRISMA_METRICS_COLLECTION_INTERVAL_SECONDS
- Add **kwargs to mock _patched_get_or_create_gauge for forward compat
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
