When scope_user_search_to_org flag is ON, team admins (non-org-admins) were
getting 403 because the code only checked for ORG_ADMIN role in org memberships.
Now checks all org memberships (any role) and falls back to the API key's team_id
to resolve the org.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add SecretRedactionFilter to scrub API keys, tokens, and credentials
from all log records (messages, args, tracebacks, extra fields).
- Enable redaction by default; opt out with LITELLM_DISABLE_REDACT_SECRETS=true
- Redact patterns: sk-*, Bearer tokens, x-api-key values, base64 creds
- Handle JSON formatter exception hooks and percent-style format args
- Snapshot dict iteration to avoid RuntimeError during concurrent logging
Asserts that GET /get/internal_user_settings returns
INTERNAL_USER_VIEW_ONLY on a fresh DB with no saved settings,
matching the runtime fallback in SSO/SCIM/JWT provisioning.
The Pydantic default for user_role was INTERNAL_USER, but all runtime
provisioning paths (SSO, SCIM, JWT) fall back to INTERNAL_USER_VIEW_ONLY
when no settings are saved. This caused the UI to show "Internal User"
on fresh instances while new users actually got "Internal Viewer".
The second `responses_api_bridge_check` call unconditionally overwrote
`responses_api_model_info` set by the first call. For models using the
`responses/` prefix (e.g. `azure/responses/<deployment>`), the first
check correctly detected `mode: "responses"` and stripped the prefix,
but the second check then overwrote it with an empty dict since the
model no longer started with `responses/`.
Skip the second check when the first already detected responses mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add sagemaker_nova provider for Nova models on SageMaker
Add support for custom/fine-tuned Amazon Nova models (Nova Micro, Nova Lite,
Nova 2 Lite) deployed on SageMaker Inference real-time endpoints.
Nova uses OpenAI-compatible request/response format with additional
Nova-specific parameters (top_k, reasoning_effort, allowed_token_ids,
truncate_prompt_tokens) and requires stream:true in the request body.
Nova endpoints also reject 'model' in the request body.
Changes:
- New provider: sagemaker_nova/<endpoint-name>
- SagemakerNovaConfig inherits from SagemakerChatConfig
- Override transform_request to strip 'model' from request body
- Override supports_stream_param_in_request_body (True for Nova)
- Extend get_supported_openai_params with Nova-specific params
- Refactored SagemakerChatConfig to use custom_llm_provider param
instead of hardcoded strings (backwards-compatible)
- Consolidated main.py routing for sagemaker_chat and sagemaker_nova
- 22 unit tests + 9 integration tests (skip-gated)
- Documentation with SDK, streaming, multimodal, and proxy examples
- All tests verified against live SageMaker Nova endpoint
* fix: move integration tests to tests/local_testing/ per test directory policy
* fix: remove unused module-level SagemakerNovaConfig instance
The sagemaker_nova_config singleton was never imported or used — the
ProviderConfigManager creates its own instance via the lambda registered
in utils.py. Removing this leftover boilerplate.
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* docs: add Claude Code skills page for litellm-skills
* docs: move skills page to new 'Manage with AI Agents' section
* docs: simplify install to one-liner, rename to LiteLLM Skills
- Fix mypy arg-type error in background_streaming.py by adding proper
type annotation and cast for terminal_status
- Fix ruff F401 false positive for httpx import in vantage_destination.py
caused by from __future__ import annotations
- Fix flaky test_arouter_responses_api_bridge by providing a properly
structured mock response to prevent exception mapping errors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The default timeout (600s) from AsyncHTTPHandler is sufficient.
Removes the explicit timeout param to keep the call simple.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-request `httpx.AsyncClient` with `get_async_httpx_client`
to avoid the +500ms latency penalty from creating new clients per
request. Updates tests to mock the cached client factory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The documentation test checks that all env vars used in code are
documented. The Vantage integration added 5 new env vars without
updating the reference table.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix genuine regression in responses_api_bridge_check where the second
call assigned to `model_info` instead of `responses_api_model_info`,
preventing gpt-5.4 + tools + reasoning_effort from routing to the
Responses API bridge.
Also update outdated tests:
- Vantage tests: match "csv" file key and use supported column names
- Anthropic caching test: add "type": "custom" to expected tool payload
- Claude Agent SDK test: remove non-deterministic LLM content assertion
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Reduce RENDER_BATCH_SIZE from 5 to 3 for more frequent chart updates
- Add LoadingOutlined spinner at the start of all fetching Alert banners
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace subtle loading text with antd Alert banners that clearly communicate
pagination status, and batch state flushes to reduce chart re-renders.
- Replace inline loading text with warning Alert banners showing progress,
"open a new tab" link with ExportOutlined icon, and primary Stop button
- Batch setData calls every 5 pages instead of per-page to cut re-renders ~80%
- Reduce fetch delay from 500ms to 300ms for faster data loading
- Add "Charts will update periodically" messaging to set expectations
- Fix pre-existing TS error: Button icon prop was using render function instead of ReactNode
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Map provider finish_reason "content_filtered" to the OpenAI-compatible "content_filter" and extend core_helpers tests to cover this case.
Made-with: Cursor
* fix: restore offline tiktoken cache for non-root envs
Made-with: Cursor
* chore: mkdir for custom tiktoken cache dir
Made-with: Cursor
* test: patch tiktoken.get_encoding in custom-dir test to avoid network
Made-with: Cursor
* test: clear CUSTOM_TIKTOKEN_CACHE_DIR in helper for test isolation
Made-with: Cursor
* test: restore default_encoding module state after custom-dir test
Made-with: Cursor
1. Replace ...args spread in useEffect deps with JSON.stringify(args) key
to prevent infinite re-renders when callers pass unstable array references.
2. Add missing agentCancelled partial-data message in EntityUsage so the
outer condition no longer renders an empty div.
3. Store setTimeout ID in a ref and clearTimeout on cleanup/cancel to avoid
orphaned timers under rapid re-renders.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test's partial vi.mock of @/components/networking was missing the daily
activity call exports now imported by EntityUsage via ENTITY_FETCH_FNS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(bedrock): respect s3_region_name for batch file uploads (#23569)
* fix(bedrock): respect s3_region_name for batch file uploads (GovCloud fix)
* fix: s3_region_name always wins over aws_region_name for S3 signing (Greptile feedback)
* fix: _filter_headers_for_aws_signature - Bedrock KB (#23571)
* fix: _filter_headers_for_aws_signature
* fix: filter None header values in all post-signing re-merge paths
Addresses Greptile feedback: None-valued headers were being filtered
during SigV4 signing but re-merged back into the final headers dict
afterward, which would cause downstream HTTP client failures.
Made-with: Cursor
* feat(router): tag_regex routing — route by User-Agent regex without per-developer tag config (#23594)
* feat(router): add tag_regex support for header-based routing
Adds a new `tag_regex` field to litellm_params that lets operators route
requests based on regex patterns matched against request headers — primarily
User-Agent — without requiring per-developer tag configuration.
Use case: route all Claude Code traffic (User-Agent: claude-code/x.y.z) to
a dedicated deployment by setting:
tag_regex:
- "^User-Agent: claude-code\\/"
in the deployment's litellm_params. Works alongside existing `tags` routing;
exact tag match takes precedence over regex match. Unmatched requests fall
through to deployments tagged `default`.
The matched deployment, pattern, and user_agent are recorded in
`metadata["tag_routing"]` so they flow through to SpendLogs automatically.
* fix(tag_regex): address backwards-compat, metadata overwrite, and warning noise
Three issues from code review:
1. Backwards-compat: `has_tag_filter` was widened to activate on any non-empty
User-Agent, which would raise ValueError for existing deployments using plain
tags without a `default` fallback. Fix: only activate header-based regex
filtering when at least one candidate deployment has `tag_regex` configured.
2. Metadata overwrite: `metadata["tag_routing"]` was overwritten for every
matching deployment in the loop, leaving inaccurate provenance when multiple
deployments match. Fix: write only for the first match.
3. Warning noise: an invalid regex pattern logged one warning per header string
rather than once per pattern. Fix: compile first (catching re.error once),
then iterate over header strings.
Also adds two new tests covering these cases, and adds docs page for
tag_regex routing with a Claude Code walk-through.
* refactor(tag_regex): remove unnecessary _healthy_list copy
* docs: merge tag_regex section into tag_routing.md, remove standalone page
- Add ## Regex-based tag routing (tag_regex) section to existing
tag_routing.md instead of a separate page
- Remove tag_regex_routing.md standalone doc (odd UX to have a separate
page for a sub-feature)
- Remove proxy/tag_regex_routing from sidebars.js
- Add match_any=False debug warning in tag_based_routing.py when regex
routing fires under strict mode (regex always uses OR semantics)
* fix(tag_regex): address greptile review - security docs, strict-mode enforcement, validation order
- Strengthen security note in tag_routing.md: explicitly state User-Agent
is client-supplied and can be set to any value; frame tag_regex as a
traffic classification hint, not an access-control mechanism
- Move tag_regex startup validation before _add_deployment() so an invalid
pattern never leaves partial router state
- Enforce match_any=False strict-tag policy: when a deployment has both
tags and tag_regex and the strict tag check fails, skip the regex fallback
rather than silently bypassing the operator's intent
- Extract per-deployment match logic into _match_deployment() helper to
keep get_deployments_for_tag() readable
- Add two new tests: strict-mode blocks regex fallback, regex-only
deployment still matches under match_any=False
* fix(ci): apply Black formatting to 14 files and stabilize flaky caplog tests
- Run Black formatter on 14 files that were failing the lint check
- Replace caplog-based assertions in TestAliasConflicts with
unittest.mock.patch on verbose_logger.warning for xdist compatibility
- The caplog fixture can produce empty text in pytest-xdist workers
in certain CI environments, causing flaky test failures
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
