The second `responses_api_bridge_check` call unconditionally overwrote
`responses_api_model_info` set by the first call. For models using the
`responses/` prefix (e.g. `azure/responses/<deployment>`), the first
check correctly detected `mode: "responses"` and stripped the prefix,
but the second check then overwrote it with an empty dict since the
model no longer started with `responses/`.
Skip the second check when the first already detected responses mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add sagemaker_nova provider for Nova models on SageMaker
Add support for custom/fine-tuned Amazon Nova models (Nova Micro, Nova Lite,
Nova 2 Lite) deployed on SageMaker Inference real-time endpoints.
Nova uses OpenAI-compatible request/response format with additional
Nova-specific parameters (top_k, reasoning_effort, allowed_token_ids,
truncate_prompt_tokens) and requires stream:true in the request body.
Nova endpoints also reject 'model' in the request body.
Changes:
- New provider: sagemaker_nova/<endpoint-name>
- SagemakerNovaConfig inherits from SagemakerChatConfig
- Override transform_request to strip 'model' from request body
- Override supports_stream_param_in_request_body (True for Nova)
- Extend get_supported_openai_params with Nova-specific params
- Refactored SagemakerChatConfig to use custom_llm_provider param
instead of hardcoded strings (backwards-compatible)
- Consolidated main.py routing for sagemaker_chat and sagemaker_nova
- 22 unit tests + 9 integration tests (skip-gated)
- Documentation with SDK, streaming, multimodal, and proxy examples
- All tests verified against live SageMaker Nova endpoint
* fix: move integration tests to tests/local_testing/ per test directory policy
* fix: remove unused module-level SagemakerNovaConfig instance
The sagemaker_nova_config singleton was never imported or used — the
ProviderConfigManager creates its own instance via the lambda registered
in utils.py. Removing this leftover boilerplate.
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* docs: add Claude Code skills page for litellm-skills
* docs: move skills page to new 'Manage with AI Agents' section
* docs: simplify install to one-liner, rename to LiteLLM Skills
- Fix mypy arg-type error in background_streaming.py by adding proper
type annotation and cast for terminal_status
- Fix ruff F401 false positive for httpx import in vantage_destination.py
caused by from __future__ import annotations
- Fix flaky test_arouter_responses_api_bridge by providing a properly
structured mock response to prevent exception mapping errors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The default timeout (600s) from AsyncHTTPHandler is sufficient.
Removes the explicit timeout param to keep the call simple.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-request `httpx.AsyncClient` with `get_async_httpx_client`
to avoid the +500ms latency penalty from creating new clients per
request. Updates tests to mock the cached client factory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The documentation test checks that all env vars used in code are
documented. The Vantage integration added 5 new env vars without
updating the reference table.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix genuine regression in responses_api_bridge_check where the second
call assigned to `model_info` instead of `responses_api_model_info`,
preventing gpt-5.4 + tools + reasoning_effort from routing to the
Responses API bridge.
Also update outdated tests:
- Vantage tests: match "csv" file key and use supported column names
- Anthropic caching test: add "type": "custom" to expected tool payload
- Claude Agent SDK test: remove non-deterministic LLM content assertion
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Reduce RENDER_BATCH_SIZE from 5 to 3 for more frequent chart updates
- Add LoadingOutlined spinner at the start of all fetching Alert banners
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace subtle loading text with antd Alert banners that clearly communicate
pagination status, and batch state flushes to reduce chart re-renders.
- Replace inline loading text with warning Alert banners showing progress,
"open a new tab" link with ExportOutlined icon, and primary Stop button
- Batch setData calls every 5 pages instead of per-page to cut re-renders ~80%
- Reduce fetch delay from 500ms to 300ms for faster data loading
- Add "Charts will update periodically" messaging to set expectations
- Fix pre-existing TS error: Button icon prop was using render function instead of ReactNode
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Map provider finish_reason "content_filtered" to the OpenAI-compatible "content_filter" and extend core_helpers tests to cover this case.
Made-with: Cursor
* fix: restore offline tiktoken cache for non-root envs
Made-with: Cursor
* chore: mkdir for custom tiktoken cache dir
Made-with: Cursor
* test: patch tiktoken.get_encoding in custom-dir test to avoid network
Made-with: Cursor
* test: clear CUSTOM_TIKTOKEN_CACHE_DIR in helper for test isolation
Made-with: Cursor
* test: restore default_encoding module state after custom-dir test
Made-with: Cursor
1. Replace ...args spread in useEffect deps with JSON.stringify(args) key
to prevent infinite re-renders when callers pass unstable array references.
2. Add missing agentCancelled partial-data message in EntityUsage so the
outer condition no longer renders an empty div.
3. Store setTimeout ID in a ref and clearTimeout on cleanup/cancel to avoid
orphaned timers under rapid re-renders.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test's partial vi.mock of @/components/networking was missing the daily
activity call exports now imported by EntityUsage via ENTITY_FETCH_FNS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(bedrock): respect s3_region_name for batch file uploads (#23569)
* fix(bedrock): respect s3_region_name for batch file uploads (GovCloud fix)
* fix: s3_region_name always wins over aws_region_name for S3 signing (Greptile feedback)
* fix: _filter_headers_for_aws_signature - Bedrock KB (#23571)
* fix: _filter_headers_for_aws_signature
* fix: filter None header values in all post-signing re-merge paths
Addresses Greptile feedback: None-valued headers were being filtered
during SigV4 signing but re-merged back into the final headers dict
afterward, which would cause downstream HTTP client failures.
Made-with: Cursor
* feat(router): tag_regex routing — route by User-Agent regex without per-developer tag config (#23594)
* feat(router): add tag_regex support for header-based routing
Adds a new `tag_regex` field to litellm_params that lets operators route
requests based on regex patterns matched against request headers — primarily
User-Agent — without requiring per-developer tag configuration.
Use case: route all Claude Code traffic (User-Agent: claude-code/x.y.z) to
a dedicated deployment by setting:
tag_regex:
- "^User-Agent: claude-code\\/"
in the deployment's litellm_params. Works alongside existing `tags` routing;
exact tag match takes precedence over regex match. Unmatched requests fall
through to deployments tagged `default`.
The matched deployment, pattern, and user_agent are recorded in
`metadata["tag_routing"]` so they flow through to SpendLogs automatically.
* fix(tag_regex): address backwards-compat, metadata overwrite, and warning noise
Three issues from code review:
1. Backwards-compat: `has_tag_filter` was widened to activate on any non-empty
User-Agent, which would raise ValueError for existing deployments using plain
tags without a `default` fallback. Fix: only activate header-based regex
filtering when at least one candidate deployment has `tag_regex` configured.
2. Metadata overwrite: `metadata["tag_routing"]` was overwritten for every
matching deployment in the loop, leaving inaccurate provenance when multiple
deployments match. Fix: write only for the first match.
3. Warning noise: an invalid regex pattern logged one warning per header string
rather than once per pattern. Fix: compile first (catching re.error once),
then iterate over header strings.
Also adds two new tests covering these cases, and adds docs page for
tag_regex routing with a Claude Code walk-through.
* refactor(tag_regex): remove unnecessary _healthy_list copy
* docs: merge tag_regex section into tag_routing.md, remove standalone page
- Add ## Regex-based tag routing (tag_regex) section to existing
tag_routing.md instead of a separate page
- Remove tag_regex_routing.md standalone doc (odd UX to have a separate
page for a sub-feature)
- Remove proxy/tag_regex_routing from sidebars.js
- Add match_any=False debug warning in tag_based_routing.py when regex
routing fires under strict mode (regex always uses OR semantics)
* fix(tag_regex): address greptile review - security docs, strict-mode enforcement, validation order
- Strengthen security note in tag_routing.md: explicitly state User-Agent
is client-supplied and can be set to any value; frame tag_regex as a
traffic classification hint, not an access-control mechanism
- Move tag_regex startup validation before _add_deployment() so an invalid
pattern never leaves partial router state
- Enforce match_any=False strict-tag policy: when a deployment has both
tags and tag_regex and the strict tag check fails, skip the regex fallback
rather than silently bypassing the operator's intent
- Extract per-deployment match logic into _match_deployment() helper to
keep get_deployments_for_tag() readable
- Add two new tests: strict-mode blocks regex fallback, regex-only
deployment still matches under match_any=False
* fix(ci): apply Black formatting to 14 files and stabilize flaky caplog tests
- Run Black formatter on 14 files that were failing the lint check
- Replace caplog-based assertions in TestAliasConflicts with
unittest.mock.patch on verbose_logger.warning for xdist compatibility
- The caplog fixture can produce empty text in pytest-xdist workers
in certain CI environments, causing flaky test failures
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Previously, EntityUsage only fetched page 1 of paginated daily spend endpoints,
showing incomplete data. UsagePageView fetched all pages but blocked the UI until
completion. This adds a reusable usePaginatedDailyActivity hook that fetches pages
sequentially with 500ms delays, updates charts progressively, and supports
cancellation on unmount or user action.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When bridging /chat/completions to the Responses API, content items with
type 'file' were falling through to the default handler and being
stringified as input_text. This caused the model to receive the Python
dict representation as plain text instead of the actual file content.
Add explicit handling for type 'file' that correctly maps:
{"type": "file", "file": {"file_data": "...", "filename": "..."}}
to:
{"type": "input_file", "file_data": "...", "filename": "..."}
FixesBerriAI/litellm#23588
Rewrite Default Team Settings UI from Tremor to antd with hardcoded fields,
fix default team params not applying during team creation or persisting
across proxy restarts, remove dead code, and add comprehensive tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(security): bump tar to 7.5.11 and tornado to 6.5.5
- tar >=7.5.11: fixes CVE-2026-31802 (HIGH) in node-pkg
- tornado >=6.5.5: fixes CVE-2026-31958 (HIGH) and GHSA-78cv-mqj4-43f7 (MEDIUM) in python-pkg
Addresses vulnerabilities found in ghcr.io/berriai/litellm:main-v1.82.0-stable Trivy scan.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: document tar override is enforced via Dockerfile, not npm
* fix: revert invalid JSON comment in package.json tar override
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Handle response.failed, response.incomplete, and response.cancelled terminal events in background streaming
Previously the background streaming task only handled response.completed and
hardcoded the final status to "completed". This missed three other terminal
event types from the OpenAI streaming spec, causing failed/incomplete/cancelled
responses to be incorrectly marked as completed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Remove unused terminal_response_data variable
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Address code review: derive fallback status from event type, rewrite tests as integration tests
1. Replace hardcoded "completed" fallback in response_data.get("status")
with _event_to_status lookup so that response.incomplete and
response.cancelled events get the correct fallback if the response
body ever omits the status field.
2. Replace duplicated-logic unit tests with integration tests that
exercise background_streaming_task directly using mocked streaming
responses and assert on the final update_state call arguments.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Remove dead mock_processor and unused mock_response parameter from test helper
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Remove FastAPI and UserAPIKeyAuth imports from test file
These types were only used as Mock(spec=...) arguments. Drop the spec
constraints and remove the top-level imports to avoid pulling FastAPI
into test files outside litellm/proxy/.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Log warning when streaming response has no body_iterator
If base_process_llm_request returns a non-streaming response (no
body_iterator), log a warning since this likely indicates a
misconfiguration or provider error rather than a successful completion.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
