Greptile: async_post_call_streaming_iterator_hook omitted request_data on OUTPUT
make_bedrock_api_request calls, so standard_logging attached to a throwaway dict.
- Pass request_data for parallel OUTPUT task and OUTPUT-only branch.
- Add unit tests asserting OUTPUT (and parallel INPUT) receive the same request_data.
Made-with: Cursor
- Add redact_nested_match_and_regex_keys in core_helpers for nested match/regex.
- Apply in CustomGuardrail standard logging; Bedrock forwards raw JSON to avoid double redaction.
- Delegate Bedrock HTTP detail assessments and _redact_pii_matches to the same helper.
- Extend unit tests (core_helpers, CustomGuardrail, Bedrock spend-log mock).
Made-with: Cursor
The projected-spend alert in _update_key_cache read from
existing_spend_obj.litellm_budget_table["soft_budget"], but the nested
dict is never populated for virtual keys (the combined_view SQL maps
budget fields to flat top-level attributes instead). This made the
check dead code — it silently short-circuited on every request, and
when unblocked, crashed update_cache with a Pydantic ValidationError
because _get_projected_spend_over_limit returns a date object but
CallInfo.projected_exceeded_date expects str.
Fixes: read from the flat existing_spend_obj.soft_budget field that IS
populated, and stringify projected_exceeded_date.
Also marks team soft budget email alerts as enterprise in docs.
Closes#20324
* [Test] Add Azure async chat completion timeout test. WIP
* Capture TTFT for /v1/messages streaming responses
The pass-through streaming path for /v1/messages (Anthropic, Bedrock,
Vertex AI, Azure AI, Minimax) logged completion_start_time only after
the entire stream finished. async_success_handler then fell back to
end_time, making TTFT equal to total duration or null in the UI and
Prometheus.
Record the timestamp of the first chunk in async_sse_wrapper and
propagate it to model_call_details before the logging handler runs,
so gen_ai.response.time_to_first_token reflects the real first-chunk
latency.
Fixes#25598
* [Refactor] Implement timeout resolution logic in completion function
add fetch ``request_timeout`` from litellm_settings
* remove stale test case
* remove extra print statement
* default request timeout value in constants to 600s to match timeout defaults handled in the proxy
* fix request timeout if using default value from constants.py
* update code structure, test cases
* only override if the global timeout sets timeout to 6000s
* update code structure, move hard coded values to const and make the reslve function readable by moving fallback logic to a seperate function
* modify default timeout values, replacing hard coded ones with default values defined
---------
Co-authored-by: harish876 <harishgokul01@gmail.com>
Co-authored-by: Joaquin Hui Gomez <joaquinhuigomez@users.noreply.github.com>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 30, 8) (push) Waiting to run
* fix: make PodLockManager.release_lock atomic compare-and-delete
Re-lands #21226 (reverted in #21469).
release_lock() previously did GET + compare + DEL in separate calls,
leaving a window where another pod could reacquire the lock between
the GET and DEL, causing a stale owner to delete a live lock.
Fix: use a Redis Lua script for atomic compare-and-delete. Script
registration is cached per PodLockManager instance. Falls back to
the old GET+DEL path for cache backends that don't expose
async_register_script.
Original revert was due to e2e tests running in CI without Redis.
Those tests now carry @pytest.mark.skip(reason="Requires Redis connection.")
so this re-land is safe.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: add Lua fallback on execution error + test coverage gaps
Address Greptile review feedback on #24466:
1. Wrap Lua script execution in try/except — if Redis clears loaded
scripts (restart) or scripting is disabled, fall back to GET+DEL
rather than letting the exception propagate and leave the lock held
until TTL. Reset cached script handle so the next call re-registers.
2. Add test_release_lock_lua_path_emits_released_event — verifies
_emit_released_lock_event is called when Lua path returns 1.
3. Add test_release_lock_falls_back_to_get_del_when_lua_execution_fails
— verifies the fallback path is taken and script handle is reset.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
publish_proxy_extras is superseded by PyPI trusted publishing (OIDC);
the CircleCI project no longer has PYPI_PUBLISH_* credentials configured.
prisma_schema_sync is a leaf smoke test with no dependents, and db push
against the current schema is already exercised by e2e_ui_testing.
Required test-unit-* and related workflows only triggered on PRs targeting
main, so feature PRs routed through litellm_internal_staging or
litellm_oss_branch never dispatched the full suite. Branch protection
reported BLOCKED even when CircleCI was green.
Expand pull_request and push branch filters to also match
litellm_internal_staging, litellm_oss_branch, and "litellm_**" (using **
so branch names containing "/" also match).
- Assert make_bedrock_api_request forwards logging_event_type to standard logging
and legacy INPUT maps to pre_call when omitted.
- Assert during_call_hook invokes Bedrock async_moderation_hook when native path is used.
Made-with: Cursor
