Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 30, 8) (push) Waiting to run
Previously these were silently dropped with a verbose warning, which
could break observability integrations without surfacing a clear error.
Now raises ValueError with remediation steps (configure server-side
or pass the resolved value) so callers get immediate, actionable feedback.
Brings the date-range branch in line with the non-date-range branch which
already hashes sk- prefixed tokens before querying. Adds coverage for
filter-combination behavior in view_spend_logs.
- Log a warning when dropping callback params that carry os.environ/
references so operators notice the misconfiguration.
- Require absolute paths in oidc/file/ and correct the documented
example to use the leading-slash form.
- Drop the unused return value from _reject_os_environ_references.
- Reject os.environ/ references supplied via /health/test_connection
request params instead of resolving them; config-sourced values are
already resolved before reaching the endpoint.
- Skip os.environ/ references in dynamic callback params loaded from
per-request metadata.
- Constrain oidc/file/ to an allowed credential directory allowlist
(defaults to /var/run/secrets and /run/secrets, overridable via
LITELLM_OIDC_ALLOWED_CREDENTIAL_DIRS).
Budget table entries (team members, end-users) used duration_in_seconds()
for a sliding-window reset, while keys/users/teams used calendar-aligned
get_budget_reset_time(). This made "30d" and "1mo" mean different things
depending on entity type. Now both paths use get_budget_reset_time() for
consistent calendar-aligned resets (e.g. "30d" → 1st of next month).
Fixes#25432
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two code paths in key_management_endpoints.py call hash_token()
unconditionally when invalidating the user_api_key_cache after a key
update. When the caller passes a pre-hashed token ID (not an sk-
prefixed key), hash_token() double-hashes it, producing a cache key
that does not match the actual cached entry. Cache invalidation
silently fails.
This is compounded by update_cache() which writes the stale cached key
object back with a fresh 60s TTL after every successful request,
preventing natural TTL expiry. The stale entry (with outdated fields
like max_budget=None) persists indefinitely under load.
PR #24969 fixed this in update_key_fn but missed two other call sites:
- _process_single_key_update (bulk update path)
- _execute_virtual_key_regeneration (key rotation path)
Fix: replace hash_token() with _hash_token_if_needed() in both
locations, matching the pattern already used elsewhere in the file.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Renames the new per-guardrail opt-out field from `disabled_global_guardrails`
to `opted_out_global_guardrails` to eliminate the one-character collision with
the legacy `disable_global_guardrails` boolean kill switch. Adds a type guard
on the new gate so a misnamed bool can't crash the guardrail check. Filters
duplicates out of the team-edit guardrail display for legacy teams that have a
global name persisted in `metadata.guardrails` from before this PR. Drops the
unused `isGuardrailsLoading` and `guardrailsError` destructures left in
AddModelForm after the hook refactor.
Adds Python tests for the new gate behavior (root, litellm_metadata, metadata,
non-matching name, empty list, malformed bool value, opt-in coexistence) and
extends useGuardrails.test.ts to exercise the global / optional partition
logic that the rebuilt hook performs in its `select` transform.
Wires the legacy kill switch and the new opt-out list together in the team
edit form so they can never fall out of sync:
- Toggling the kill switch reactively updates the Guardrails Select via
`onValuesChange` — switch on strips all globals from the selection, switch
off re-adds them. Existing opt-in extras are preserved either way.
- When the switch is on, global options in the Select are individually
disabled (greyed out) so the user can still manage opt-in guardrails but
cannot accidentally re-enable a global the kill switch is bypassing.
- The save handler writes both fields together: `disable_global_guardrails`
reflects the switch, and `opted_out_global_guardrails` is set to either
every global (when the switch is on) or the user's explicit opt-outs.
- `effectiveGuardrails` for the form's initialValues honors the kill switch
on legacy teams so the form opens in a state that matches what the runtime
gate is actually doing — fixes the visual lie where chips appeared active
while the switch was bypassing them.
The backend gate already reads the list as the primary path with the bool
as a fallback, so untouched legacy teams keep working until they get edited,
at which point they migrate naturally.
Rename disable_global_guardrail → disable_global_guardrails to match
the key name used by litellm_pre_call_utils.py, the API endpoints,
and the UI when propagating key/team metadata.
The singular form was introduced in PR #16983 and has never matched
the plural form written by the rest of the codebase, so the feature
silently did nothing.
Re-applies fix originally from #25488. Original commit could not be
merged due to missing signature.
Co-Authored-By: Remi Mabon <remi.mabon@redcare-pharmacy.com>
- Add _verify_org_access to deprecated POST /organization/info endpoint
- Move get_user_object to module-level import in organization_endpoints.py
- Add tests for _verify_team_access 403 denial path
- Introduced a new method in `FileContentStreamingHandler` to resolve streaming request parameters, enhancing the routing logic based on credentials.
- Updated the `should_stream_file_content` method to check against supported providers.
- Cleaned up type hints and imports across multiple files for better organization and clarity.
- Added comprehensive tests to validate the new routing behavior and ensure original data integrity during streaming requests.
* fix(s3): add retry with exponential backoff for transient S3 503/500 errors
S3 occasionally returns 503 "Slow Down" during PUT operations when
request rates spike above partition limits. The current code makes a
single upload attempt via httpx — unlike boto3, httpx has no built-in
retry for transient S3 errors. Failed uploads permanently lose the
request's audit/logging data.
Add exponential backoff retry (3 attempts, 1s/2s delays) for S3
500/503 responses in both async_upload_data_to_s3 and
upload_data_to_s3. Logs a warning on each retry with the S3 object
key for observability.
In production we observed ~18 permanent S3 upload failures per day
(124 over 7 days) — all transient 503s that would have succeeded on
a single retry.
* test(s3): add unit tests for S3 upload retry logic
Tests cover:
- Async retry on 503 (succeeds on second attempt)
- Async retry on 500
- Exhausted retries on persistent 503 (calls handle_callback_failure)
- No retry on 4xx errors (403)
- Sync retry on 503
* style(s3): move time import to module level
Address review feedback: move `import time` from inside
upload_data_to_s3 to the top-level imports per project style guide.
* fix(logging): preserve proxy key-auth metadata on /v1/messages Langfuse traces
update_from_kwargs() overwrites proxy metadata (user_api_key_hash, etc.)
with Anthropic's native metadata when both exist. Merge instead of replace.
* fix(test): update stale assertion for new metadata merge semantics
* test: add explicit conflict-resolution test for metadata merge
* feat(containers): Azure container routing, managed IDs, and delete response wire format
- Add AzureContainerConfig and safe URL joining for paths with api-version query
- Encode/decode managed container IDs in responses, streaming, and proxy handlers
- Accept OpenAI delete response object literal container.file.deleted
- Tests for Azure URL regression and DeleteContainerFileResponse parsing
Made-with: Cursor
* fix(responses): gate response id update on parsed_chunk having response
Delta stream events do not include a response body; Mock-based tests
(and any truthy synthetic .response on transforms) must not trigger
_update_responses_api_response_id_with_model_id. Fixes
test_stop_async_iteration_not_logged_as_failure (TypeError: Mock not iterable).
Made-with: Cursor
* feat(containers): encode container IDs in SDK responses for routing
- Add ContainerRequestUtils.encode_container_id_in_response utility
- Encode container_id in create/retrieve/delete responses (SDK path)
- Fix streaming iterator: gate response ID update on parsed_chunk key
- Follows responses API pattern (encode after handler, not in handler)
Made-with: Cursor
* fix(containers): module-level imports and managed cntr_ ID encoding
- Move ResponsesAPIRequestUtils imports to module scope (utils, main, handler_factory).
- Serialize absent model_id as empty segment instead of literal None; decode empty
and legacy "None" segments as missing for router affinity.
- Add unit tests for build/decode round-trip and legacy IDs.
Made-with: Cursor
* fix(containers): decode managed IDs in endpoint_factory SDK path
- Add decode_managed_container_id_for_request in containers/utils and reuse from main.
- Strip LiteLLM cntr_ wrappers before generic_container_handler (64-char API limit).
- Resolve provider for logging/errors; add unit test for decode helper.
- Use resolved_custom_llm_provider after decode for mypy-safe provider typing.
Made-with: Cursor
* Fix p1 concern
* Fix p1 concern
1. exclude-newer: change from absolute "2026-04-10" to relative "3 days".
All pinned deps were published before the 3-day cutoff. Re-locked so
uv lock --check passes in test-mcp.yml and test-linting.yml.
2. test_eager_tiktoken_load: run all 10 env var values in a single
subprocess instead of spawning 10 separate processes. Each cold
import litellm takes ~78s on CI, so the old loop took ~13 min on a
single xdist worker. Now takes ~78s total.
3. proxy-db remaining timeout: increase from 20 to 30 minutes. The
remaining group has 51 test files and was consistently timing out at
71% across all branches (pre-existing issue, not migration-related).
When modify_params is true, Bedrock Converse setup no longer prepends or
appends the default user message if the boundary assistant turn has
prefix: true, so OpenAI-style assistant prefill reaches the API unchanged.
Made-with: Cursor
* feat(guardrails): optional skip system message in unified guardrail inputs
Made-with: Cursor
* feat(dashboard): skip_system_message_in_guardrail in guardrail UI
Add a tri-state control (inherit / yes / no) when creating or editing
guardrails so admins can set litellm_params.skip_system_message_in_guardrail
without YAML. Table edit merges existing litellm_params before PUT to avoid
wiping content-filter and other provider fields.
Document the dashboard flow in the guardrails quick start with a screenshot.
Made-with: Cursor
* fix(guardrails): type structured_messages as AllMessageValues for mypy
Use AllMessageValues in openai_messages_without_system and cast adapter
request messages so GenericGuardrailAPIInputs matches TypedDict.
Made-with: Cursor
* fix(responses): map refusal stop_reason to incomplete status in streaming
Fixes streaming responses API translation where Anthropic's stop_reason="refusal"
was incorrectly translated to status="completed" instead of "incomplete".
Root cause: build_base_response was unconditionally overwriting finish_reason
with None from later chunks, losing the terminal content_filter value.
Changes:
- streaming_chunk_builder_utils: skip None finish_reason values in build_base_response
- streaming_iterator: snapshot chunks before returning pending events (sync path)
- streaming_handler: treat usage-only chunks as meaningful content
- transformation: map finish_reason=refusal to status=incomplete
- tests: add regression tests for refusal handling
Made-with: Cursor
* Fix test
