- Fix __main__ block: test returns None now, so always exited 1
- Close litellm_async_client in test_ssl_verification_with_aiohttp_transport
- Save/restore litellm.force_ipv4 and litellm.disable_aiohttp_transport
in test_force_ipv4_transport and test_aiohttp_disabled_transport
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
count_aiohttp_sessions() iterates every object in the Python GC,
which hangs in CI when xdist workers have millions of loaded objects.
Replace with direct session.closed checks — same coverage, no hang.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove real HTTP call to example.com in test_force_ipv4_transport
(hangs in CI when network is slow/unavailable)
- Close leaked aiohttp.ClientSession in test_ssl_verification_with_aiohttp_transport
- Add cleanup for transports in test_aiohttp_transport_trust_env_setting
- Add cleanup for handler in test_ssl_security_level
- Add cleanup for transport in test_ssl_context_transport
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prisma migrate deploy failures on pre-existing instances
Fixes failed migrations due to idempotent schema changes on pre-existing litellm instances.
Problems:
1. P3018 recovery handler never returned True on successful resolution, causing "Database setup failed after multiple retries" even when the final recovery succeeded
2. _roll_back_migration exceptions escaped the P3018 handler, preventing _resolve_specific_migration from running
3. Migration SQL used ADD COLUMN/DROP COLUMN without IF [NOT] EXISTS, failing if schema was already modified
Changes:
- Add return True after successful P3018 idempotent error recovery
- Wrap _roll_back_migration in try/except to allow recovery continuation even if rollback fails
- Make migration.sql idempotent with IF NOT EXISTS / IF EXISTS clauses
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* test: add migration SQL idempotency safety tests
Adds TestMigrationSQLIdempotency test class that statically validates all
migration SQL files created after 2026-03-11 use idempotent DDL:
- ADD COLUMN must use IF NOT EXISTS
- DROP COLUMN must use IF EXISTS
- DROP INDEX must use IF EXISTS
- CREATE INDEX must use IF NOT EXISTS
This prevents the class of errors where prisma migrate deploy fails on
pre-existing instances because the schema was already modified.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: also catch TimeoutExpired in P3018 rollback handler
_roll_back_migration uses subprocess.run with timeout=60, so it can raise
subprocess.TimeoutExpired in addition to CalledProcessError. Without
catching this, a slow database during rollback would escape the handler
and bypass _resolve_specific_migration — the same class of bug.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: make all 85 migration SQL files idempotent, remove test cutoff
Fixed all existing migration files to use IF [NOT] EXISTS for DDL
statements (ADD COLUMN, DROP COLUMN, DROP INDEX, CREATE INDEX).
Removed the date cutoff from the idempotency tests so they now
validate all migrations, not just recent ones.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: make migration failure non-fatal by default, add --require_db_migration flag
By default the proxy now warns and continues when database migration
fails. Pass --require_db_migration (or set REQUIRE_DB_MIGRATION=true)
to restore the previous behavior of exiting with an error.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: wrap _resolve_specific_migration in try/except, guard RENAME COLUMN and ADD CONSTRAINT
Three fixes:
1. _resolve_specific_migration in the P3018 handler was not wrapped in
try/except, so failures there would bypass the return True and
propagate unexpectedly — partially defeating the rollback fix.
2. Bare RENAME COLUMN in 20260303000000_update_tool_table_policies was
non-idempotent. Wrapped in DO $$ IF EXISTS block. Also wrapped all
28 bare ADD CONSTRAINT statements across 9 migration files in
DO $$ IF NOT EXISTS (pg_constraint) blocks.
3. Added test_rename_column_is_guarded and test_add_constraint_is_guarded
to TestMigrationSQLIdempotency for full DDL coverage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: retry after resolving idempotent migration, guard DROP CONSTRAINT
Three fixes:
1. Both P3009 and P3018 idempotent handlers returned True after
resolving a single migration, exiting before remaining pending
migrations were applied. Now they continue the retry loop so
prisma migrate deploy runs again for any remaining migrations.
2. Two migration files had bare DROP CONSTRAINT without a DO $$ IF
EXISTS guard, which fails if the constraint was already dropped.
Wrapped both in idempotent DO $$ blocks.
3. Added test_drop_constraint_is_guarded to catch unguarded DROP
CONSTRAINT in future migrations.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: P3009 try/except, CREATE TABLE IF NOT EXISTS, restore fail-fast default
Four fixes:
1. P3009 idempotent handler now has the same try/except around
_roll_back_migration and _resolve_specific_migration as the P3018
handler. Previously a rollback or resolve failure in the P3009 path
would propagate and leave the migration unresolved.
2. Added IF NOT EXISTS to all 57 bare CREATE TABLE statements across
34 migration files. Added test_create_table_uses_if_not_exists to
catch this pattern.
3. Reverted the backwards-incompatible default behavior change: the
proxy now fails fast on migration failure (original behavior).
Added --skip_db_migration_check / SKIP_DB_MIGRATION_CHECK to
opt into warn-and-continue instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
When scope_user_search_to_org flag is ON, team admins (non-org-admins) were
getting 403 because the code only checked for ORG_ADMIN role in org memberships.
Now checks all org memberships (any role) and falls back to the API key's team_id
to resolve the org.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add SecretRedactionFilter to scrub API keys, tokens, and credentials
from all log records (messages, args, tracebacks, extra fields).
- Enable redaction by default; opt out with LITELLM_DISABLE_REDACT_SECRETS=true
- Redact patterns: sk-*, Bearer tokens, x-api-key values, base64 creds
- Handle JSON formatter exception hooks and percent-style format args
- Snapshot dict iteration to avoid RuntimeError during concurrent logging
* feat: add sagemaker_nova provider for Nova models on SageMaker
Add support for custom/fine-tuned Amazon Nova models (Nova Micro, Nova Lite,
Nova 2 Lite) deployed on SageMaker Inference real-time endpoints.
Nova uses OpenAI-compatible request/response format with additional
Nova-specific parameters (top_k, reasoning_effort, allowed_token_ids,
truncate_prompt_tokens) and requires stream:true in the request body.
Nova endpoints also reject 'model' in the request body.
Changes:
- New provider: sagemaker_nova/<endpoint-name>
- SagemakerNovaConfig inherits from SagemakerChatConfig
- Override transform_request to strip 'model' from request body
- Override supports_stream_param_in_request_body (True for Nova)
- Extend get_supported_openai_params with Nova-specific params
- Refactored SagemakerChatConfig to use custom_llm_provider param
instead of hardcoded strings (backwards-compatible)
- Consolidated main.py routing for sagemaker_chat and sagemaker_nova
- 22 unit tests + 9 integration tests (skip-gated)
- Documentation with SDK, streaming, multimodal, and proxy examples
- All tests verified against live SageMaker Nova endpoint
* fix: move integration tests to tests/local_testing/ per test directory policy
* fix: remove unused module-level SagemakerNovaConfig instance
The sagemaker_nova_config singleton was never imported or used — the
ProviderConfigManager creates its own instance via the lambda registered
in utils.py. Removing this leftover boilerplate.
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
- Fix mypy arg-type error in background_streaming.py by adding proper
type annotation and cast for terminal_status
- Fix ruff F401 false positive for httpx import in vantage_destination.py
caused by from __future__ import annotations
- Fix flaky test_arouter_responses_api_bridge by providing a properly
structured mock response to prevent exception mapping errors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-request `httpx.AsyncClient` with `get_async_httpx_client`
to avoid the +500ms latency penalty from creating new clients per
request. Updates tests to mock the cached client factory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix genuine regression in responses_api_bridge_check where the second
call assigned to `model_info` instead of `responses_api_model_info`,
preventing gpt-5.4 + tools + reasoning_effort from routing to the
Responses API bridge.
Also update outdated tests:
- Vantage tests: match "csv" file key and use supported column names
- Anthropic caching test: add "type": "custom" to expected tool payload
- Claude Agent SDK test: remove non-deterministic LLM content assertion
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Map provider finish_reason "content_filtered" to the OpenAI-compatible "content_filter" and extend core_helpers tests to cover this case.
Made-with: Cursor
* fix: restore offline tiktoken cache for non-root envs
Made-with: Cursor
* chore: mkdir for custom tiktoken cache dir
Made-with: Cursor
* test: patch tiktoken.get_encoding in custom-dir test to avoid network
Made-with: Cursor
* test: clear CUSTOM_TIKTOKEN_CACHE_DIR in helper for test isolation
Made-with: Cursor
* test: restore default_encoding module state after custom-dir test
Made-with: Cursor
* fix(bedrock): respect s3_region_name for batch file uploads (#23569)
* fix(bedrock): respect s3_region_name for batch file uploads (GovCloud fix)
* fix: s3_region_name always wins over aws_region_name for S3 signing (Greptile feedback)
* fix: _filter_headers_for_aws_signature - Bedrock KB (#23571)
* fix: _filter_headers_for_aws_signature
* fix: filter None header values in all post-signing re-merge paths
Addresses Greptile feedback: None-valued headers were being filtered
during SigV4 signing but re-merged back into the final headers dict
afterward, which would cause downstream HTTP client failures.
Made-with: Cursor
* feat(router): tag_regex routing — route by User-Agent regex without per-developer tag config (#23594)
* feat(router): add tag_regex support for header-based routing
Adds a new `tag_regex` field to litellm_params that lets operators route
requests based on regex patterns matched against request headers — primarily
User-Agent — without requiring per-developer tag configuration.
Use case: route all Claude Code traffic (User-Agent: claude-code/x.y.z) to
a dedicated deployment by setting:
tag_regex:
- "^User-Agent: claude-code\\/"
in the deployment's litellm_params. Works alongside existing `tags` routing;
exact tag match takes precedence over regex match. Unmatched requests fall
through to deployments tagged `default`.
The matched deployment, pattern, and user_agent are recorded in
`metadata["tag_routing"]` so they flow through to SpendLogs automatically.
* fix(tag_regex): address backwards-compat, metadata overwrite, and warning noise
Three issues from code review:
1. Backwards-compat: `has_tag_filter` was widened to activate on any non-empty
User-Agent, which would raise ValueError for existing deployments using plain
tags without a `default` fallback. Fix: only activate header-based regex
filtering when at least one candidate deployment has `tag_regex` configured.
2. Metadata overwrite: `metadata["tag_routing"]` was overwritten for every
matching deployment in the loop, leaving inaccurate provenance when multiple
deployments match. Fix: write only for the first match.
3. Warning noise: an invalid regex pattern logged one warning per header string
rather than once per pattern. Fix: compile first (catching re.error once),
then iterate over header strings.
Also adds two new tests covering these cases, and adds docs page for
tag_regex routing with a Claude Code walk-through.
* refactor(tag_regex): remove unnecessary _healthy_list copy
* docs: merge tag_regex section into tag_routing.md, remove standalone page
- Add ## Regex-based tag routing (tag_regex) section to existing
tag_routing.md instead of a separate page
- Remove tag_regex_routing.md standalone doc (odd UX to have a separate
page for a sub-feature)
- Remove proxy/tag_regex_routing from sidebars.js
- Add match_any=False debug warning in tag_based_routing.py when regex
routing fires under strict mode (regex always uses OR semantics)
* fix(tag_regex): address greptile review - security docs, strict-mode enforcement, validation order
- Strengthen security note in tag_routing.md: explicitly state User-Agent
is client-supplied and can be set to any value; frame tag_regex as a
traffic classification hint, not an access-control mechanism
- Move tag_regex startup validation before _add_deployment() so an invalid
pattern never leaves partial router state
- Enforce match_any=False strict-tag policy: when a deployment has both
tags and tag_regex and the strict tag check fails, skip the regex fallback
rather than silently bypassing the operator's intent
- Extract per-deployment match logic into _match_deployment() helper to
keep get_deployments_for_tag() readable
- Add two new tests: strict-mode blocks regex fallback, regex-only
deployment still matches under match_any=False
* fix(ci): apply Black formatting to 14 files and stabilize flaky caplog tests
- Run Black formatter on 14 files that were failing the lint check
- Replace caplog-based assertions in TestAliasConflicts with
unittest.mock.patch on verbose_logger.warning for xdist compatibility
- The caplog fixture can produce empty text in pytest-xdist workers
in certain CI environments, causing flaky test failures
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Rewrite Default Team Settings UI from Tremor to antd with hardcoded fields,
fix default team params not applying during team creation or persisting
across proxy restarts, remove dead code, and add comprehensive tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Handle response.failed, response.incomplete, and response.cancelled terminal events in background streaming
Previously the background streaming task only handled response.completed and
hardcoded the final status to "completed". This missed three other terminal
event types from the OpenAI streaming spec, causing failed/incomplete/cancelled
responses to be incorrectly marked as completed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Remove unused terminal_response_data variable
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Address code review: derive fallback status from event type, rewrite tests as integration tests
1. Replace hardcoded "completed" fallback in response_data.get("status")
with _event_to_status lookup so that response.incomplete and
response.cancelled events get the correct fallback if the response
body ever omits the status field.
2. Replace duplicated-logic unit tests with integration tests that
exercise background_streaming_task directly using mocked streaming
responses and assert on the final update_state call arguments.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Remove dead mock_processor and unused mock_response parameter from test helper
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Remove FastAPI and UserAPIKeyAuth imports from test file
These types were only used as Mock(spec=...) arguments. Drop the spec
constraints and remove the top-level imports to avoid pulling FastAPI
into test files outside litellm/proxy/.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Log warning when streaming response has no body_iterator
If base_process_llm_request returns a non-streaming response (no
body_iterator), log a warning since this likely indicates a
misconfiguration or provider error rather than a successful completion.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* PR #22867 added _remove_scope_from_cache_control for Bedrock and Azure AI but omitted Vertex AI. This applies the same pattern to VertexAIPartnerModelsAnthropicMessagesConfig."
* PR #22867 added _remove_scope_from_cache_control for Bedrock and Azure AI but omitted Vertex AI. This applies the same pattern to VertexAIPartnerModelsAnthropicMessagesConfig."
* PR #22867 added _remove_scope_from_cache_control to AzureAnthropicMessagesConfig
but missed VertexAIPartnerModelsAnthropicMessagesConfi Rather than duplicating the method again, moved it up to the base AnthropicMessagesConfig so all providers
inherit it, and removed the now-redundant copy from the Azure AI subclass.
* PR #22867 added _remove_scope_from_cache_control to AzureAnthropicMessagesConfig
but missed VertexAIPartnerModelsAnthropicMessagesConfi Rather than duplicating the method again, moved it up to the base AnthropicMessagesConfig so all providers
inherit it, and removed the now-redundant copy from the Azure AI subclass.
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Use `min`/`max` instead of `_min`/`_max` for Prisma group_by input
parameters. The underscore-prefixed names are output keys, not input
kwargs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Verifies that validate_key_team_change does not call can_team_access_model
for the "all-team-models" sentinel, allowing keys with that value to be
moved between teams without model validation failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tests were outdated after #23472 added pagination (take/order) to find_many
and stale-row cleanup via update_many. Updated assertions to match new call
signatures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The self-exclusion filter compared raw key strings against SHA-256
hashed tokens from the DB, so keys were never excluded and
double-counting persisted. Now hash data.key before comparison.
Also add tpm_limit_type/rpm_limit_type to _throughput_fields_changed
guard, fall back to existing_key_row.team_id for team limit checks
(matching the org pattern), and add team self-exclusion test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Only run org validation (get_org_object + _check_org_key_limits) when
the update actually touches throughput-related fields (tpm_limit,
rpm_limit, or organization_id). Previously, any update to a key
belonging to an org would trigger the check, which would fail with a
400 if the org had been deleted — blocking unrelated field changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When updating a key, _check_org_key_limits and _check_team_key_limits
would include the key being updated in the find_many results, causing
its current limits to be counted twice (once from the DB query, once
from the new requested limits). This caused false 400 errors on valid
limit adjustments.
Fix: exclude the key being updated (by matching token) from the
allocated totals before checking limits.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Instead of hardcoding SPEND_PER_REQUEST (which broke when the model
changed from gpt-3.5-turbo-0301 to gpt-3.5-turbo), make a single
calibration request first, poll for its spend, and use that as the
per-request cost. Fails fast with pytest.fail() after 5 retries if
calibration cannot determine the cost.
Also fixes a bug in test_basic_spend_accuracy where the user spend
assertion error message referenced user_info['info'] instead of
user_info['user_info'].
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Increase wait timeout to 90s and pytest.fail() instead of silently
continuing, so the failure message points at the real cause.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gpt-3.5-turbo-0301 was removed from the model cost map, so every call
had response_cost=0 and team member spend never increased. The wait
helper also returned True after 3s regardless of whether spend updated.
- Switch fake-openai-endpoint to gpt-3.5-turbo (has pricing in cost map)
- Remove premature early-return in wait_for_team_member_spend_update
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Increase max_retries from 6 to 9 and retry_delay from 10s to 20s
(180s total wait, up from 60s) to give batch cost tracking more time
to finish before cleanup attempts file deletion.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
