- Remove re-raise in _store_transactions_in_redis so one Redis
push failure doesn't drop remaining transaction types
- Downgrade per-push success log from info to debug to reduce noise
- Fix misleading error message in update_database — entity spend
updates run as independent tasks and are not affected by this catch
* fix(router): preserve _hidden_params in FallbackStreamWrapper so x-litellm-overhead-duration-ms is emitted for streaming requests
* test(router): add regression test for FallbackStreamWrapper _hidden_params preservation
Set prometheus_emit_stream_label: true in litellm_settings to emit a
stream label (True/False/None) on litellm_proxy_total_requests_metric.
Opt-in to avoid breaking cardinality on existing deployments.
Add a Python script that detects duplicate issues using title similarity
(difflib.SequenceMatcher) and closes them via the gh CLI. Two-tier system:
- 0.6 threshold: informational comment via existing wow-actions step
- 0.85 threshold: auto-close with comment, label, and not_planned reason
Includes a workflow_dispatch workflow for one-time batch scans and
integrates auto-close into the existing check_duplicate_issues workflow
for newly opened issues.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Elevate silent debug-level and bare except:pass error paths to
warning/error so spend tracking failures are visible in production logs.
All new log messages are prefixed with "Spend tracking -" for easy
filtering. Changes cover the full request-to-DB lifecycle:
enqueue, in-memory flush, Redis buffer push/pop, DB commit,
cache updates, spend log writes, and pod lock management.
Also fixes a copy-paste bug in _update_team_cache that logged
"end user" instead of "team".
Replace 11 separate asyncio.create_task() calls per request with a
single batched task that runs all spend-update helpers sequentially.
This reduces task scheduling overhead at high RPS (11,000 -> 1,000
tasks/sec at 1K RPS) and cuts 5 copy.deepcopy(payload) calls to 1
shared copy.
Also fixes a mutation bug where the daily agent spend handler received
the raw payload without deepcopy, unlike all other daily helpers.
Verify that extra_headers are correctly forwarded to OpenAI's
images.generate() in both sync and async paths, and that they
are absent when not provided.
Add headers parameter to image_generation() and aimage_generation() methods
in OpenAI provider, and pass headers from images/main.py to ensure custom
headers like cf-aig-authorization are properly forwarded to the OpenAI API.
Aligns behavior with completion() method and Azure provider implementation.
* staged first pass
* black
* Update litellm/proxy/health_check.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* simpler
* restore cached logo
* fix tests for perform_health_check max_concurrency arg
* implement pr suggestion
* and the helm chart
* add configureable resources and probes to the deployment in the helm chart
* more helm chart unittests
* move some background healthcheck loggin to debug
---------
Co-authored-by: Sean Glover <sglover@athenahealth.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
# Please enter a commit message to explain why this merge is necessary,
# especially if it merges an updated upstream into a topic branch.
#
# Lines starting with '#' will be ignored, and an empty message aborts
# the commit.
Replace racy check-then-increment RPM logic with atomic increment-first
pattern to prevent concurrent requests from bypassing the rate limit.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Non-owner Internal Users could see and interact with the "Edit Settings"
button in the key Settings tab for keys they don't own. The button was
gated by `rolesWithWriteAccess.includes(userRole)` (role-only check)
instead of `canModifyKey` (ownership-aware), unlike the Regenerate and
Delete buttons which already used the correct check.
Replace the condition with `canModifyKey` so the Edit Settings button
follows the same proxy-admin / team-admin / key-owner logic as the
other action buttons. Add tests covering all permission paths.
- Wrap onboarding page with QueryClientProvider to prevent runtime crash
(mirrors the same pattern used in LoginPage)
- Stage deletion of stale litellm/ui/litellm-dashboard/src/app/onboarding/page.tsx
committed at the wrong path
- Rename all 16 test names to start with "should" per AGENTS.md convention
* feat(realtime): add guardrail hook for voice transcription in Realtime API
Adds a new `realtime_input_transcription` guardrail event hook that fires
after Whisper transcription completes, before the LLM generates a response.
When a guardrail blocks, a synthetic warning is sent to the client and
`response.create` is never forwarded — the LLM never responds.
Also rewrites `create_response: true` → `false` in client `session.update`
so the proxy controls when responses are triggered.
* feat(realtime): speak guardrail block message as audio via TTS
Instead of sending synthetic text events when a guardrail blocks,
send response.create with forced instructions so OpenAI's TTS speaks
the warning message — user hears the block instead of just seeing text.
* fix(realtime): speak exact content filter error message via TTS
Extract the human-readable error string from HTTPException.detail
so the spoken warning says e.g. "Content blocked: keyword 'system update'
detected" instead of the raw str(e) repr.
* fix(realtime): reliably enforce create_response=false for guardrails
- Proxy now injects session.update with create_response=false immediately
on session.created (when guardrails are active), instead of rewriting
the client's session.update — works regardless of what the client sends
- Add response.cancel before the warning response.create to kill any
in-flight LLM response that snuck through before the guardrail fired
* refactor(realtime): call apply_guardrail directly, remove dedicated hook method
The async_realtime_input_transcription_hook in CustomGuardrail and
ContentFilterGuardrail was just a thin wrapper that called apply_guardrail —
the same interface used by /chat and /messages. Remove the wrapper and call
apply_guardrail directly from run_realtime_guardrails, keeping the pattern
consistent across all endpoints.
* docs: add Realtime API guardrails tutorial and flow diagram
* fix: address Greptile review comments
- Forward user_api_key_dict through realtime_api/main.py (_arealtime) so
it actually reaches RealTimeStreaming instead of always being None
- Run guardrail interception in provider_config path too (e.g. Gemini),
not only the OpenAI direct path
- Narrow exception catch to HTTPException/ValueError only; re-raise
unexpected errors so programming bugs surface in logs rather than
silently appearing as guardrail blocks
- Update tests: mock apply_guardrail directly (hook method was removed),
replace session.update client-rewrite test with session.created
injection test matching the new server-side approach
* fix: address latest Greptile review comments
- Remove fastapi import from SDK-layer file; check for status_code/detail
attrs instead to identify guardrail-block exceptions vs programming errors
- Add store_message() before continue in transcription interception so
transcription events are logged in the non-provider_config path
- Inject create_response=false on session.created in provider_config path
(Gemini etc.) to match the OpenAI path — prevents LLM auto-responding
before guardrail runs on VAD-detected turns
Adds YAML topic category files for military_status, disability, age_discrimination,
religion, and gender_sexual_orientation to block employment discrimination prompts
like "Do not hire veterans because they may have mental health issues."
Previously these were not blocked because:
- The prebuilt regex patterns used strict \b word boundaries that didn't match
plurals (veterans, disabilities, Muslims)
- gender_sexual_orientation pattern was LGBTQ+-focused and missed women/female
- age_discrimination pattern missed "over 50" phrasing
- No conditional (identifier + discriminatory intent) detection existed for these
protected classes
Each new YAML file uses the bias_racial.yaml pattern: identifier_words (protected
class terms) + additional_block_words (discriminatory employment actions), plus
always_block_keywords for explicit discriminatory phrases. Exceptions prevent false
positives for legitimate diversity programs, accommodation discussions, etc.
Also fixes regex plurals in patterns.json: veterans?, disabilit(y|ies), muslims?,
adds wom[ae]n?/females? to gender pattern, and over\s+\d+ to age pattern.
Evals: 100% precision/recall/F1/accuracy on all 5 new categories (89 total cases,
0 FP, 0 FN). Existing insults and investment evals unaffected.
