Greptile: async_post_call_streaming_iterator_hook omitted request_data on OUTPUT
make_bedrock_api_request calls, so standard_logging attached to a throwaway dict.
- Pass request_data for parallel OUTPUT task and OUTPUT-only branch.
- Add unit tests asserting OUTPUT (and parallel INPUT) receive the same request_data.
Made-with: Cursor
- Add redact_nested_match_and_regex_keys in core_helpers for nested match/regex.
- Apply in CustomGuardrail standard logging; Bedrock forwards raw JSON to avoid double redaction.
- Delegate Bedrock HTTP detail assessments and _redact_pii_matches to the same helper.
- Extend unit tests (core_helpers, CustomGuardrail, Bedrock spend-log mock).
Made-with: Cursor
- Assert make_bedrock_api_request forwards logging_event_type to standard logging
and legacy INPUT maps to pre_call when omitted.
- Assert during_call_hook invokes Bedrock async_moderation_hook when native path is used.
Made-with: Cursor
Bedrock ApplyGuardrail uses INPUT/OUTPUT for the API body; spend logs must use
the proxy hook (pre_call, during_call, post_call). Pass logging_event_type from
each hook into make_bedrock_api_request.
During-call was still logged as pre_call because unified guardrails call
apply_guardrail with input_type=request. BedrockGuardrail now sets
use_native_during_call_hook so during_call runs async_moderation_hook instead.
Includes a small test asserting the Bedrock class flag.
Made-with: Cursor
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 30, 8) (push) Waiting to run
The suite was superseded by ui/litellm-dashboard/e2e_tests/ on 2026-04-08
and is no longer referenced by CircleCI, docs, or Makefile targets. Drop
the directory wholesale and remove the orphaned e2e:psql npm script that
pointed at its runner.
Previously these were silently dropped with a verbose warning, which
could break observability integrations without surfacing a clear error.
Now raises ValueError with remediation steps (configure server-side
or pass the resolved value) so callers get immediate, actionable feedback.
Brings the date-range branch in line with the non-date-range branch which
already hashes sk- prefixed tokens before querying. Adds coverage for
filter-combination behavior in view_spend_logs.
- Log a warning when dropping callback params that carry os.environ/
references so operators notice the misconfiguration.
- Require absolute paths in oidc/file/ and correct the documented
example to use the leading-slash form.
- Drop the unused return value from _reject_os_environ_references.
- Reject os.environ/ references supplied via /health/test_connection
request params instead of resolving them; config-sourced values are
already resolved before reaching the endpoint.
- Skip os.environ/ references in dynamic callback params loaded from
per-request metadata.
- Constrain oidc/file/ to an allowed credential directory allowlist
(defaults to /var/run/secrets and /run/secrets, overridable via
LITELLM_OIDC_ALLOWED_CREDENTIAL_DIRS).
Renames the new per-guardrail opt-out field from `disabled_global_guardrails`
to `opted_out_global_guardrails` to eliminate the one-character collision with
the legacy `disable_global_guardrails` boolean kill switch. Adds a type guard
on the new gate so a misnamed bool can't crash the guardrail check. Filters
duplicates out of the team-edit guardrail display for legacy teams that have a
global name persisted in `metadata.guardrails` from before this PR. Drops the
unused `isGuardrailsLoading` and `guardrailsError` destructures left in
AddModelForm after the hook refactor.
Adds Python tests for the new gate behavior (root, litellm_metadata, metadata,
non-matching name, empty list, malformed bool value, opt-in coexistence) and
extends useGuardrails.test.ts to exercise the global / optional partition
logic that the rebuilt hook performs in its `select` transform.
Wires the legacy kill switch and the new opt-out list together in the team
edit form so they can never fall out of sync:
- Toggling the kill switch reactively updates the Guardrails Select via
`onValuesChange` — switch on strips all globals from the selection, switch
off re-adds them. Existing opt-in extras are preserved either way.
- When the switch is on, global options in the Select are individually
disabled (greyed out) so the user can still manage opt-in guardrails but
cannot accidentally re-enable a global the kill switch is bypassing.
- The save handler writes both fields together: `disable_global_guardrails`
reflects the switch, and `opted_out_global_guardrails` is set to either
every global (when the switch is on) or the user's explicit opt-outs.
- `effectiveGuardrails` for the form's initialValues honors the kill switch
on legacy teams so the form opens in a state that matches what the runtime
gate is actually doing — fixes the visual lie where chips appeared active
while the switch was bypassing them.
The backend gate already reads the list as the primary path with the bool
as a fallback, so untouched legacy teams keep working until they get edited,
at which point they migrate naturally.
Rename disable_global_guardrail → disable_global_guardrails to match
the key name used by litellm_pre_call_utils.py, the API endpoints,
and the UI when propagating key/team metadata.
The singular form was introduced in PR #16983 and has never matched
the plural form written by the rest of the codebase, so the feature
silently did nothing.
Re-applies fix originally from #25488. Original commit could not be
merged due to missing signature.
Co-Authored-By: Remi Mabon <remi.mabon@redcare-pharmacy.com>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 30, 8) (push) Has been cancelled
The test expected "stream": false in the serialized request body, but
stream is only included in optional_params when explicitly passed by
the caller. litellm.completion() defaults stream=None which is excluded
from non_default_params. Assert individual fields instead of the full
serialized JSON to avoid brittleness around optional field inclusion.
usage-based-routing tracks RPM in log_success_event which runs in a
background ThreadPoolExecutor. The cache update races with the next
call's routing check in both sync (tight loop) and async (concurrent
gather) modes, making over-limit detection non-deterministic.
Skip the over-limit parametrization (num_try_send > num_allowed_send).
The under-limit cases (2 sent, 3 allowed) still verify the routing
strategy works. Rate-limit enforcement is properly tested with mocks
in tests/test_litellm/test_router/test_enforce_model_rate_limits.py.
- Add _verify_org_access to deprecated POST /organization/info endpoint
- Move get_user_object to module-level import in organization_endpoints.py
- Add tests for _verify_team_access 403 denial path
- Introduced a new method in `FileContentStreamingHandler` to resolve streaming request parameters, enhancing the routing logic based on credentials.
- Updated the `should_stream_file_content` method to check against supported providers.
- Cleaned up type hints and imports across multiple files for better organization and clarity.
- Added comprehensive tests to validate the new routing behavior and ensure original data integrity during streaming requests.
* fix(s3): add retry with exponential backoff for transient S3 503/500 errors
S3 occasionally returns 503 "Slow Down" during PUT operations when
request rates spike above partition limits. The current code makes a
single upload attempt via httpx — unlike boto3, httpx has no built-in
retry for transient S3 errors. Failed uploads permanently lose the
request's audit/logging data.
Add exponential backoff retry (3 attempts, 1s/2s delays) for S3
500/503 responses in both async_upload_data_to_s3 and
upload_data_to_s3. Logs a warning on each retry with the S3 object
key for observability.
In production we observed ~18 permanent S3 upload failures per day
(124 over 7 days) — all transient 503s that would have succeeded on
a single retry.
* test(s3): add unit tests for S3 upload retry logic
Tests cover:
- Async retry on 503 (succeeds on second attempt)
- Async retry on 500
- Exhausted retries on persistent 503 (calls handle_callback_failure)
- No retry on 4xx errors (403)
- Sync retry on 503
* style(s3): move time import to module level
Address review feedback: move `import time` from inside
upload_data_to_s3 to the top-level imports per project style guide.
* fix(logging): preserve proxy key-auth metadata on /v1/messages Langfuse traces
update_from_kwargs() overwrites proxy metadata (user_api_key_hash, etc.)
with Anthropic's native metadata when both exist. Merge instead of replace.
* fix(test): update stale assertion for new metadata merge semantics
* test: add explicit conflict-resolution test for metadata merge
* feat(containers): Azure container routing, managed IDs, and delete response wire format
- Add AzureContainerConfig and safe URL joining for paths with api-version query
- Encode/decode managed container IDs in responses, streaming, and proxy handlers
- Accept OpenAI delete response object literal container.file.deleted
- Tests for Azure URL regression and DeleteContainerFileResponse parsing
Made-with: Cursor
* fix(responses): gate response id update on parsed_chunk having response
Delta stream events do not include a response body; Mock-based tests
(and any truthy synthetic .response on transforms) must not trigger
_update_responses_api_response_id_with_model_id. Fixes
test_stop_async_iteration_not_logged_as_failure (TypeError: Mock not iterable).
Made-with: Cursor
* feat(containers): encode container IDs in SDK responses for routing
- Add ContainerRequestUtils.encode_container_id_in_response utility
- Encode container_id in create/retrieve/delete responses (SDK path)
- Fix streaming iterator: gate response ID update on parsed_chunk key
- Follows responses API pattern (encode after handler, not in handler)
Made-with: Cursor
* fix(containers): module-level imports and managed cntr_ ID encoding
- Move ResponsesAPIRequestUtils imports to module scope (utils, main, handler_factory).
- Serialize absent model_id as empty segment instead of literal None; decode empty
and legacy "None" segments as missing for router affinity.
- Add unit tests for build/decode round-trip and legacy IDs.
Made-with: Cursor
* fix(containers): decode managed IDs in endpoint_factory SDK path
- Add decode_managed_container_id_for_request in containers/utils and reuse from main.
- Strip LiteLLM cntr_ wrappers before generic_container_handler (64-char API limit).
- Resolve provider for logging/errors; add unit test for decode helper.
- Use resolved_custom_llm_provider after decode for mypy-safe provider typing.
Made-with: Cursor
* Fix p1 concern
* Fix p1 concern
1. exclude-newer: change from absolute "2026-04-10" to relative "3 days".
All pinned deps were published before the 3-day cutoff. Re-locked so
uv lock --check passes in test-mcp.yml and test-linting.yml.
2. test_eager_tiktoken_load: run all 10 env var values in a single
subprocess instead of spawning 10 separate processes. Each cold
import litellm takes ~78s on CI, so the old loop took ~13 min on a
single xdist worker. Now takes ~78s total.
3. proxy-db remaining timeout: increase from 20 to 30 minutes. The
remaining group has 51 test files and was consistently timing out at
71% across all branches (pre-existing issue, not migration-related).
When modify_params is true, Bedrock Converse setup no longer prepends or
appends the default user message if the boundary assistant turn has
prefix: true, so OpenAI-style assistant prefill reaches the API unchanged.
Made-with: Cursor
* feat(guardrails): optional skip system message in unified guardrail inputs
Made-with: Cursor
* feat(dashboard): skip_system_message_in_guardrail in guardrail UI
Add a tri-state control (inherit / yes / no) when creating or editing
guardrails so admins can set litellm_params.skip_system_message_in_guardrail
without YAML. Table edit merges existing litellm_params before PUT to avoid
wiping content-filter and other provider fields.
Document the dashboard flow in the guardrails quick start with a screenshot.
Made-with: Cursor
* fix(guardrails): type structured_messages as AllMessageValues for mypy
Use AllMessageValues in openai_messages_without_system and cast adapter
request messages so GenericGuardrailAPIInputs matches TypedDict.
Made-with: Cursor
* fix(responses): map refusal stop_reason to incomplete status in streaming
Fixes streaming responses API translation where Anthropic's stop_reason="refusal"
was incorrectly translated to status="completed" instead of "incomplete".
Root cause: build_base_response was unconditionally overwriting finish_reason
with None from later chunks, losing the terminal content_filter value.
Changes:
- streaming_chunk_builder_utils: skip None finish_reason values in build_base_response
- streaming_iterator: snapshot chunks before returning pending events (sync path)
- streaming_handler: treat usage-only chunks as meaningful content
- transformation: map finish_reason=refusal to status=incomplete
- tests: add regression tests for refusal handling
Made-with: Cursor
* Fix test
1. Cap requires-python to <3.14 — no deps ship 3.14 wheels yet, and
uv's cross-version resolver fails on the Python 3.14 split.
2. Change exclude-newer from relative "30 days" to absolute "2026-04-10"
so the lockfile stays reproducible. The relative date caused
cryptography==46.0.7 (published April 8) to fall outside the window.
3. Parametrize test_eager_loading_env_var_values instead of looping —
with xdist the 6 subprocess cases can run in parallel instead of all
running sequentially on one worker (~13 min → ~2 min).
Also removed redundant case variants (Yes/YES/On/ON) that test the
same str_to_bool code path.
1. check_code_and_doc_quality: install helm before running helm lint
(Docker image cimg/python:3.11 doesn't include helm)
2. proxy_store_model_in_db_tests: skip MCP tests on Python < 3.10
since MCP module isn't available and functions inside the
`if MCP_AVAILABLE:` block don't exist as module-level attributes
