The /v1/messages/count_tokens proxy endpoint was only passing `messages`
to provider token counting APIs, discarding `system` and `tools`. This
caused clients like Claude Code to receive artificially low token counts
(e.g. 10 instead of 531), preventing proper context window management
and leading to context overflow errors.
Pass system and tools through the full chain:
- TokenCountRequest → proxy_server → provider counters → API handlers
- Bedrock: transform tools to toolConfig format, system to text blocks
- Anthropic/Azure AI: pass through directly (same API format)
Backend - Spend Log Storage for Realtime Calls:
- Collect user voice transcripts and text input during WebSocket sessions
- Store collected messages in spend logs when store_prompts_in_spend_logs enabled
- Capture tool definitions from session.update and tool calls from response.done
- Enrich proxy_server_request with tools and response with tool_calls for UI
Backend - WebSocket Auth:
- Support browser-based auth via Sec-WebSocket-Protocol subprotocol
- Echo back subprotocol on WebSocket accept
UI - Realtime Playground:
- New RealtimePlayground component with WebSocket voice+text chat
- Mic recording (PCM16 24kHz), server VAD, audio playback, text input
- Handle binary WebSocket frames (Blob/ArrayBuffer decoding)
- Add /v1/realtime endpoint option to playground endpoint selector
UI - Tools Section for Realtime Logs:
- Extract tool calls from realtime response format (response.tool_calls
and response.results[].response.output[].type=function_call)
Tests:
- 15 new backend tests for realtime streaming and spend log storage
- 4 new UI tests for realtime tool call extraction
Fixes pre-existing build errors:
- ToolPolicies.tsx: duplicate import, antd styles type
- create_key_button.tsx: missing message import
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* docs: document x-litellm-tags header and request body tags parameter
- Add documentation for x-litellm-tags header (comma-separated or array)
- Add documentation for tags in request body
- Clarify that dynamic tags override config tags
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* docs: consolidate tag documentation and improve cross-references
- Make request_tags.md the single source of truth for all tag options
- Add cross-reference from cost_tracking.md to request_tags.md
- Document both direct tags and metadata.tags formats
- Add key/team tag setup and custom header tracking to request_tags.md
- Reduce duplication and make navigation clearer
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* docs: use generic examples instead of specific company names
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* docs: clarify x-litellm-tags header format is comma-separated string
HTTP headers are always strings, not arrays. Remove misleading
array format documentation for the header parameter.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Update docs/my-website/docs/proxy/request_tags.md
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Replace literal Base64-encoded example value in docstring with a
placeholder to prevent GitGuardian/ggshield from flagging it as a
leaked Basic Authentication credential in container scans.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(proxy): add max_iterations limiter for agent session loops (#22058)
Adds a new proxy hook that enforces a per-session cap on the number of
LLM calls an agentic loop can make. Callers send a session_id with each
request, and the hook counts calls per session, returning 429 when the
configured max_iterations limit is exceeded.
- Uses Redis Lua script for atomic increment (multi-instance safe)
- Falls back to in-memory cache when Redis unavailable
- Follows parallel_request_limiter_v3 pattern
- Configurable via key metadata: {"max_iterations": 25}
- Session counters auto-expire via TTL (default 1hr)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add new code execution dataset
* feat(agent_endpoints/): allow giving agents keys
* fix: ui fixes
* feat: allow assigning mcp servers to agents
* fix: eliminate duplicate DB queries in MCP agent auth and N+1 in agent listing (#22110)
- Extract _get_agent_object_permission helper so _get_allowed_mcp_servers_for_agent
and _get_agent_tool_permissions_for_server share a single DB fetch instead of
each independently querying the same agent row (was 1+N queries per MCP request)
- Use include={"object_permission": True} on find_many in get_all_agents_from_db
to eagerly load permissions in one query instead of N+1
- Use include={"object_permission": True} on create/update/find_unique in all
agent CRUD operations, removing attach_object_permission_to_dict follow-up calls
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Add new document explaining automatic credential usage tracking and tagging. When models use reusable credentials, LiteLLM automatically injects a Credential: <name> tag on requests, enabling credential-level spend tracking on the Usage page with no additional configuration.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* feat(ui): add user filtering to usage page
Adds "User Usage" as a new view option in the usage page dropdown,
allowing admins to view and filter usage data by individual users
via the existing /user/daily/activity backend endpoint.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(ui/): working usage filtering
* fix(ui): use single-select for user filter and add tests
The user entity type's backend endpoint only accepts a single user_id,
so the filter now uses single-select mode instead of multi-select.
Added tests for the new user entity type in EntityUsage and
UsageViewSelect. Updated CLAUDE.md and AGENTS.md with guidance on
UI/backend contract consistency and test coverage for new entity types.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* revert: remove unintended package-lock.json changes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* revert: restore package-lock.json to merge base state
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Added unit tests for all pricing calculator components with 64 passing tests across 5 test files:
- multi_export_utils.test.ts (16 tests for PDF/CSV export functions)
- use_multi_cost_estimate.test.ts (15 tests for cost estimation hook)
- multi_export_dropdown.test.tsx (8 tests for export dropdown component)
- multi_cost_results.test.tsx (15 tests for results display and UI states)
- index.test.tsx (10 tests for main calculator component)
Also fixed missing page description for tool-policies page in page_metadata.ts.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Duplicate import was causing UI build to fail:
- Line 4: import { Select, Switch, Tooltip } from 'antd'
- Line 5: import { Select, Tooltip } from 'antd' (duplicate)
Removed the duplicate line 5.
Add a `request_duration_ms` column to `LiteLLM_SpendLogs` to track request
duration. New rows are computed at write time. Legacy rows use a COALESCE
fallback in the `/spend/logs/ui` query to compute duration on the fly from
`endTime - startTime`. The field is also sortable in the UI endpoint.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Google's Interactions API spec changed the status enum:
- Values are now lowercase (was uppercase)
- 'UNSPECIFIED' value was removed
Updated test to match the current spec from:
https://ai.google.dev/static/api/interactions.openapi.json
