Commit graph

2905 commits

Author SHA1 Message Date
Yuneng Jiang
77f2d8ed95
chore: fixes
Some checks failed
Unit Tests: Caching (Redis) / caching-redis (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (key-generation, tests/proxy_unit_tests/test_key_generate_prisma.py, 30, 0) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Security / security (push) Has been cancelled
2026-04-05 00:28:32 -07:00
Sameer Kankute
9605462fce Fix free models working from UI 2026-03-03 18:16:00 +05:30
yuneng-jiang
f59c164286 [Fix] Virtual Keys pagination displays stale totals when filtering
When applying a key_alias filter on the Virtual Keys page, the pagination display (total count, page count) showed stale unfiltered values. The bug was that the table component tracked total_count only from the unfiltered useKeys hook, not from the filtered API response returned by useFilterLogic.

Fixes: When selecting a key alias that matches 1 key, the UI previously showed "Showing 1-50 of 509 results" and "Page 1 of 11" instead of the correct "Showing 1-1 of 1 results" and "Page 1 of 1".

Changes:
- Added filteredTotalCount state to useFilterLogic to track the total_count from filtered API responses
- Updated VirtualKeysTable to use filteredTotalCount (when set) instead of always using the unfiltered total
- Added comprehensive tests to prevent regression of pagination display logic

Type: 🐛 Bug Fix,  Test

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-03 18:15:52 +05:30
Sameer Kankute
722d041b17 Fix UI build 2026-03-03 18:15:41 +05:30
Ishaan Jaff
61e2fdf463 feat(realtime guardrails): end_session_after_n_fails + Endpoint Settings wizard step (#22165)
* feat(realtime guardrails): end_session_after_n_fails + Endpoint Settings wizard step

Adds per-session violation thresholds and an optional endpoint-settings step
to the guardrail wizard for /v1/realtime.

Backend:
- Add end_session_after_n_fails, on_violation, realtime_violation_message fields
  to BaseLitellmParams (no DB migration — stored in existing JSON column)
- Store same fields on CustomGuardrail instance attrs
- Pass through in litellm_content_filter initializer
- Track _violation_count per RealTimeStreaming session; close backend_ws when
  on_violation=end_session OR violation count >= end_session_after_n_fails
- Use realtime_violation_message as the spoken text (falls back to guardrail
  error string if not configured)

UI (add_guardrail_form.tsx):
- Rename "Default Categories" step to "Topics"
- Add step 5 "Endpoint Settings (Optional)" for content filter guardrails
- Call type dropdown shows /v1/realtime
- Settings are in a collapsed accordion (closed by default)
- "End session after X violations" + on_violation radio + spoken message field

Tests: 2 new tests in test_realtime_streaming.py
- test_end_session_after_n_fails_closes_connection
- test_on_violation_end_session_closes_on_first_fail

* fix(test): move inline imports to module level in realtime streaming tests

* Update ui/litellm-dashboard/src/components/guardrails/add_guardrail_form.tsx

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-03 18:15:28 +05:30
Ishaan Jaff
1ed4023700 Development environment setup (#22160)
* feat: add pretty view for realtime API logs in dashboard

- Create RealtimePrettyView component that renders structured session
  config, conversation turns with transcripts, and token breakdowns
- Update PrettyMessagesView to detect realtime responses (via
  isRealtimeResponse helper) and delegate to the new component
- Session card shows model, voice, modalities, temperature, instructions
  in a collapsible panel
- Conversation turns show status, per-turn token usage, and audio/text
  transcripts with appropriate icons
- Add 24 tests for RealtimePrettyView and 3 tests for PrettyMessagesView
- All 75 LogDetailsDrawer tests pass

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* chore: remove dev_config.yaml from tracked files

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* feat: show turn count in realtime pretty view session header and output header

- Add purple 'N turns' tag to Session card header for at-a-glance turn count
- Add 'Turns: N' to the Output section header next to tokens/cost
- Extend SectionHeader to accept optional turnCount prop
- Add 3 new tests for turn count display (singular, plural, output header)

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix: address Greptile review feedback

- Remove response.audio.done and conversation.item.created from
  isRealtimeResponse() detection since the view doesn't render them;
  prevents misleading fallback for responses with only those events
- Remove dead code: index >= 0 is always true in .map() callback

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-03-03 18:15:28 +05:30
yuneng-jiang
cababa6cc2 fixing ui build 2026-03-03 18:15:28 +05:30
Krish Dholakia
6100380568 feat(add-new-block_code_execution-guardrail): prevent agent from executing code (#22154)
Adds a new block_code_execution guardrail that detects markdown fenced code blocks
in request/response content and blocks or masks them by language. Includes full
UI integration, type definitions, compliance test dataset, and 26 unit tests.

Key guardrail capabilities:
- Regex-based fenced code block detection with configurable blocked languages
- Confidence scoring with tunable threshold
- Execution-intent heuristics (request-side only) with conflict resolution
- Block or mask actions for detected code
- Support for pre_call, post_call, and during_call event hooks

Security hardening:
- Response-side blocking skips intent heuristics (LLM output doesn't contain
  user intent phrases, so checking would silently disable post_call blocking)
- No-execution short-circuit includes conflict resolution: if both no-execution
  and execution phrases match, execution intent wins
- Tightened overly broad phrases to prevent trivial bypass
- _normalize_escaped_newlines only applies to pure-escaped payloads to avoid
  corrupting content that discusses escape sequences

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:15:28 +05:30
yuneng-jiang
d1662556a9 [Feature] UI - Paginated Key Alias Select
Replace the non-paginated Key Alias filter with a new PaginatedKeyAliasSelect component that mirrors the existing PaginatedModelSelect pattern. This aligns the UI with the paginated /key/aliases endpoint from PR #22137.

Changes:
- Added useInfiniteKeyAliases hook for paginated key alias fetching
- Created PaginatedKeyAliasSelect component with infinite scroll (80% threshold)
- Updated keyAliasesCall in networking to accept page/size/search params
- Replaced Key Alias filter in Request Logs and Virtual Keys tables to use customComponent
- Removed fetchAllKeyAliases helper and related upfront fetching logic
- Added 22 tests for new component and hook; all existing tests pass (54 tests)

Fixes the issue where the UI was fetching all key aliases at once, causing performance issues with large key sets.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-03 18:15:28 +05:30
Ishaan Jaff
6c5cf440e8 feat(realtime): guardrails support for /v1/realtime WebSocket endpoint (#22152)
* feat(realtime): add guardrails query param to /v1/realtime WebSocket endpoint

- Add 'guardrails' query param (comma-separated) to realtime_websocket_endpoint
- Import websockets and websockets.exceptions at module level (fixes NameError in except clause)
- Split try/except into Phase 1 (pre-call) and Phase 2 (routing) so guardrail
  errors send back a typed error event before closing, while upstream errors
  close silently with 1011

* feat(ui): pass selectedGuardrails from sidebar to RealtimePlayground WebSocket URL

* docs(realtime): add guardrails section with dynamic passing examples
2026-03-03 18:15:28 +05:30
Krish Dholakia
2855816464 Add claims agent guardrails (5 categories + policy template) (#22113)
* Add claims agent guardrails with 243-case eval dataset

5 new category guardrails for healthcare claims agent chatbots:
- claims_fraud_coaching: fraud coaching, exaggeration, document forgery
- claims_phi_disclosure: unauthorized PHI access, bulk data extraction
- claims_prior_auth_gaming: code manipulation, medical necessity misrepresentation
- claims_system_override: system injection, rule bypass, role impersonation
- claims_medical_advice: medical advice (claims-context-aware)

Plus claims_agent_safety.yaml policy template combining all 5.

All 5 eval suites pass at 100% precision/recall/F1 (243 test cases).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add claims agent chatbot safety policy template

Combines the 5 claims guardrails into a single deployable policy template:
fraud coaching, PHI disclosure, prior-auth gaming, system override, and medical advice.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add guardrail benchmark results and UI compliance prompts

Adds benchmark results for claims, discrimination, and content filter guardrails.
Updates UI compliance prompt data.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove duplicate "file an appeal" exception in claims_prior_auth_gaming.yaml

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove unused claims_agent_safety.yaml policy template

The claims-agent-safety template in policy_templates.json references
individual category files in categories/, not this combined file.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:15:27 +05:30
yuneng-jiang
a03d2a308d [Fix] UI - MCP Servers: Make auth value optional for create flow
The backend validator and frontend form both enforced auth_value as
required when auth_type is api_key, bearer_token, or basic. Users who
want to provide auth dynamically (via per-request headers or OAuth2
flows) could not skip the field.

- Remove required validation from auth_value in create_mcp_server.tsx
  (keep whitespace-only rejection, matching the edit flow)
- Remove validate_credentials_requirements in NewMCPServerRequest
  (all downstream code already treats auth_value as optional)
- Add tests for the create MCP server component

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-03 18:15:27 +05:30
yuneng-jiang
84f239fc1b [Feature] UI - Logs: Use backend request_duration_ms and make Duration sortable
Use the backend-provided request_duration_ms field instead of computing
duration client-side from startTime/endTime. Add sort support for the
Duration column, which sends sortBy=request_duration_ms to the API.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-03 18:15:27 +05:30
Krish Dholakia
f0e93888a3 feat: add UI banner warning for detailed debug mode (#21527)
Add a prominent warning banner to the UI dashboard when detailed debug
mode (LITELLM_LOG=DEBUG) is enabled. This alerts users to significant
performance degradation caused by extensive diagnostic logging.

Backend changes:
- Enhanced /health/readiness endpoint to include log_level and
  is_detailed_debug fields
- Added detection using verbose_logger.getEffectiveLevel()
- Backward compatible - old clients ignore new fields

Frontend changes:
- Updated useHealthReadiness TypeScript interface
- Created DebugWarningBanner component using Ant Design Alert
- Integrated banner into dashboard layout below navbar
- Banner only shows when DEBUG level is active
- Non-dismissible to ensure users are aware of performance impact

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:15:27 +05:30
Ishaan Jaff
e79c160391 Realtime API: spend log storage, playground UI, tools logging, and guardrail support (#22105)
Backend - Spend Log Storage for Realtime Calls:
- Collect user voice transcripts and text input during WebSocket sessions
- Store collected messages in spend logs when store_prompts_in_spend_logs enabled
- Capture tool definitions from session.update and tool calls from response.done
- Enrich proxy_server_request with tools and response with tool_calls for UI

Backend - WebSocket Auth:
- Support browser-based auth via Sec-WebSocket-Protocol subprotocol
- Echo back subprotocol on WebSocket accept

UI - Realtime Playground:
- New RealtimePlayground component with WebSocket voice+text chat
- Mic recording (PCM16 24kHz), server VAD, audio playback, text input
- Handle binary WebSocket frames (Blob/ArrayBuffer decoding)
- Add /v1/realtime endpoint option to playground endpoint selector

UI - Tools Section for Realtime Logs:
- Extract tool calls from realtime response format (response.tool_calls
  and response.results[].response.output[].type=function_call)

Tests:
- 15 new backend tests for realtime streaming and spend log storage
- 4 new UI tests for realtime tool call extraction

Fixes pre-existing build errors:
- ToolPolicies.tsx: duplicate import, antd styles type
- create_key_button.tsx: missing message import

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-03-03 18:15:27 +05:30
Krish Dholakia
835ac240d2 Agents - assign tools (#22064)
* feat(proxy): add max_iterations limiter for agent session loops (#22058)

Adds a new proxy hook that enforces a per-session cap on the number of
LLM calls an agentic loop can make. Callers send a session_id with each
request, and the hook counts calls per session, returning 429 when the
configured max_iterations limit is exceeded.

- Uses Redis Lua script for atomic increment (multi-instance safe)
- Falls back to in-memory cache when Redis unavailable
- Follows parallel_request_limiter_v3 pattern
- Configurable via key metadata: {"max_iterations": 25}
- Session counters auto-expire via TTL (default 1hr)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add new code execution dataset

* feat(agent_endpoints/): allow giving agents keys

* fix: ui fixes

* feat: allow assigning mcp servers to agents

* fix: eliminate duplicate DB queries in MCP agent auth and N+1 in agent listing (#22110)

- Extract _get_agent_object_permission helper so _get_allowed_mcp_servers_for_agent
  and _get_agent_tool_permissions_for_server share a single DB fetch instead of
  each independently querying the same agent row (was 1+N queries per MCP request)
- Use include={"object_permission": True} on find_many in get_all_agents_from_db
  to eagerly load permissions in one query instead of N+1
- Use include={"object_permission": True} on create/update/find_unique in all
  agent CRUD operations, removing attach_object_permission_to_dict follow-up calls

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:15:17 +05:30
yuneng-jiang
0d9e6f8011 fixing build and tests 2026-03-03 18:15:17 +05:30
yuneng-jiang
f795faa640 fixing build 2026-03-03 18:15:17 +05:30
yuneng-jiang
14b785c6e6 [Test] UI - Pricing Calculator: Add comprehensive unit tests
Added unit tests for all pricing calculator components with 64 passing tests across 5 test files:
- multi_export_utils.test.ts (16 tests for PDF/CSV export functions)
- use_multi_cost_estimate.test.ts (15 tests for cost estimation hook)
- multi_export_dropdown.test.tsx (8 tests for export dropdown component)
- multi_cost_results.test.tsx (15 tests for results display and UI states)
- index.test.tsx (10 tests for main calculator component)

Also fixed missing page description for tool-policies page in page_metadata.ts.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-03 18:15:17 +05:30
Krish Dholakia
10f0f7b8c3 feat(ui): add user filtering to usage page (#22059)
* feat(ui): add user filtering to usage page

Adds "User Usage" as a new view option in the usage page dropdown,
allowing admins to view and filter usage data by individual users
via the existing /user/daily/activity backend endpoint.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(ui/): working usage filtering

* fix(ui): use single-select for user filter and add tests

The user entity type's backend endpoint only accepts a single user_id,
so the filter now uses single-select mode instead of multi-select.
Added tests for the new user entity type in EntityUsage and
UsageViewSelect. Updated CLAUDE.md and AGENTS.md with guidance on
UI/backend contract consistency and test coverage for new entity types.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* revert: remove unintended package-lock.json changes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* revert: restore package-lock.json to merge base state

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:15:17 +05:30
Shin
81377f6679 fix(ui): remove duplicate antd import in ToolPolicies
Duplicate import was causing UI build to fail:
- Line 4: import { Select, Switch, Tooltip } from 'antd'
- Line 5: import { Select, Tooltip } from 'antd' (duplicate)

Removed the duplicate line 5.
2026-03-03 18:15:17 +05:30
Krish Dholakia
f60f464753 feat: add new code execution dataset (#22065) 2026-03-03 18:15:16 +05:30
Ishaan Jaff
2b276fabbc feat(agents): assign virtual keys to agents (#22045)
* feat(agents): assign virtual keys to agents

- Add agent_id field to LiteLLM_VerificationToken (schema.prisma + _types.py)
- Pass agent_id through key generation endpoint so keys can be scoped to an agent
- Refactor Add Agent wizard to 3-step flow (Configure → Assign Key → Ready)
  - Configure: all agent fields, custom/other type with just name+description
  - Assign Key: create new key or reassign existing key to agent
  - URL is now optional for easy discovery
- Add "Agent" ownership option to Create Key modal on Virtual Keys page
  with agent selector dropdown
- Extract CreatedKeyDisplay into shared component, reused in both flows
- Add keyCreateForAgentCall networking helper
- Add test for agent_id key generation

* fix(agents): code quality fixes from self-review

- Fix test_generate_key_helper_fn_agent_id: remove bare except clause,
  use explicit assert mock_insert.called, use .kwargs for clean arg access
- Remove no-op conditional in handleNext (both branches were identical)
- Validate selectedExistingKey before calling keyUpdateCall
- Validate selectedAgentId before setting on formValues in create_key_button

* fix(ui): replace deprecated Tremor Button with Ant Design Button in CreatedKeyDisplay
2026-03-03 18:15:10 +05:30
Ishaan Jaff
9b4a2352bf fix(ui): show real tool names in logs for Anthropic format tools (#22048) 2026-03-03 18:15:10 +05:30
Alejandro Tapia
62a61f3c43 differentiate solely by model_Id 2026-03-03 18:15:10 +05:30
Alejandro Tapia
73eaa11372 healthcheck-model_id-fix: There was quite a bit of code that needed to be changed since health checks were entirely keyed by model name. This includes, proxy logic, the dashboard, and even networking, because model name was the identifier everywhere. All changed files had tests added to them, which are passing with no regressions. 2026-03-03 18:15:08 +05:30
yuneng-jiang
fd05c49787 [Feature] UI - Virtual Keys: Add KeyInfoHeader component with metadata display
Add a reusable KeyInfoHeader component to replace the inline header in KeyInfoView.
Extract LabeledField as a common component for labeled metadata display with copyable
support, default_user_id handling, and empty value placeholders.

- Migrate all icons from lucide-react/heroicons to @ant-design/icons
- Use antd native copyable with descriptive tooltips (Copy Key Alias, Copy Key ID, etc.)
- Show DefaultProxyAdminTag for default_user_id values
- Add canModifyKey, regenerateDisabled, regenerateTooltip props for permission gating
- Add tests for KeyInfoHeader (19 tests) and LabeledField (8 tests)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-03 18:15:05 +05:30
Ishaan Jaff
4b5c47f7e7 [Feat] UI - Allow using AI to understand Usage patterns (#22042)
* Add Ask AI chat component to Usage page

- Create UsageAIChatModal component with streaming chat interface
- Integrate with existing model hub for model selection
- Pass usage data context (spend, models, providers, keys) to AI
- Add Ask AI button next to Export Data button in global view
- Add tests for the new component and integration

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Convert Ask AI from modal to right-side sliding panel

- Replace UsageAIChatModal with UsageAIChatPanel
- Panel slides in from right side, usage page stays visible
- Full-height panel with header, model selector, chat area, and input
- Smooth CSS transition for open/close animation
- Update tests for new panel component (34 tests passing)

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Remove build output directory from tracking

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Add backend AI usage chat endpoint with tool calling

Backend:
- New /usage/ai/chat SSE streaming endpoint
- AI agent has get_usage_data tool that queries /user/daily/activity/aggregated
- Follows same architecture as policy AI suggest (litellm.acompletion + tools)
- Non-admin users are restricted to their own data
- 12 backend unit tests

Frontend:
- Panel now calls /usage/ai/chat backend endpoint via SSE
- Removed direct OpenAI client calls from frontend
- Added usageAiChatStream networking function following enrichPolicyTemplateStream pattern

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Make model selection optional, default to gpt-4o-mini on backend

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Add team/tag tools, status indicators, and improved AI agent

- AI agent now has 3 tools: get_usage_data, get_team_usage_data, get_tag_usage_data
- Stream status events (Thinking... Fetching... Analyzing...) to UI
- Frontend shows spinner + status text during tool execution
- Better system prompt guiding tool selection
- Entity summariser for team/tag data with ranked breakdowns
- 13 backend tests, 34 frontend tests passing

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Fix: inject today's date into system prompt so AI resolves relative dates correctly

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Show tool calls as distinct steps + render markdown in responses

- Backend emits tool_call events with tool_name, label, args, and status
- Frontend shows each tool call as a step with ✓/spinner/✗ indicator
- Tool call steps show icon, label, date range, and filters
- AI responses rendered with ReactMarkdown (bold, lists, tables, code)
- Cursor-like UX: Thinking → tool calls → Analyzing → streamed answer

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Refactor backend for code quality: proper types, constants, all functions ≤50 LOC

- TypedDict for SSE events (SSEStatusEvent, SSEToolCallEvent, etc.) and ToolHandler
- Constants for table names, entity fields, temperature, page sizes, top-N limits
- Shared _query_activity() eliminates duplicated fetch logic
- _accumulate_breakdown() + _ranked_lines() replace inline aggregation loops
- Extracted _process_tool_call() and _stream_final_response() from main stream fn
- Black + Ruff clean, all 15 functions verified ≤50 LOC
- Replaced Tremor Button with Antd Button in panel (Tremor deprecated per AGENTS.md)

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Address greptile review: security fixes and input validation

- Restrict team/tag tools to admin-only users (non-admins only get get_usage_data)
- Constrain ChatMessage.role to Literal['user', 'assistant'] to prevent system prompt injection
- Add test for base tools restriction (non-admin gets 1 tool, admin gets 3)
- Issues 3 (unused imports) and 4 (inline datetime) were already fixed in prior commit

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Address greptile round 2: sanitize errors, defense-in-depth allowlist, revert tsconfig

- Sanitize error messages: generic 'An internal error occurred' sent to client,
  full exception logged server-side via verbose_proxy_logger
- Defense-in-depth: _process_tool_call validates fn_name against role-based
  allowlist before dispatch (even though LLM only receives allowed tools)
- Revert tsconfig.json jsx back to 'preserve' (Next.js recommended default)

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Role-scoped system prompt + additional test coverage

- System prompt is now role-aware: admin sees all 3 tool descriptions,
  non-admin only sees get_usage_data (consistent with tool filtering)
- Added tests: non-admin prompt excludes team/tag tools, date injection
- 15 backend tests, 34 frontend tests passing

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Fix LLM arg validation + cap conversation size at 20 messages

- _resolve_fetch_kwargs uses .get() with ValueError for missing dates
  (handles malformed LLM tool arguments gracefully)
- MAX_CHAT_MESSAGES = 20 constant; backend truncates to last 20
- Frontend also sends only last 20 messages per request
- Prevents excessive token usage and context-length errors

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-03-03 18:15:05 +05:30
Ishaan Jaff
fef13a161b feat(proxy): tool policies - auto-discover tools + policy enforcement guardrail (#22041)
* feat(proxy): tool policies - auto-discover tools, manage policies, guardrail enforcement

- New LiteLLM_ToolTable in schema.prisma to store discovered tools
- Auto-discovery: tools seen in LLM responses get upserted via ToolDiscoveryQueue
  (hooks into DBSpendUpdateWriter, same pipeline as spend tracking)
- Management endpoints: GET /v1/tool/list, GET /v1/tool/{name}, POST /v1/tool/policy
- ToolPolicyGuardrail: blocks tool_calls in responses based on policy setting
- UI: Tool Policies page under Guardrails section with policy selector,
  filters by policy/team/key, live tail, sortable table
- Unit tests for queue, writer, endpoints, guardrail

* feat(tool-policies): track call_count + discover tools from request body and /messages API

- Add call_count column to LiteLLM_ToolTable; incremented on every flush
- Extract tools from request body too (not just response tool_calls):
  - OpenAI /chat/completions: tools[].function.name
  - Anthropic /messages pass-through: request_body.tools[].name
- Show call_count column in UI table (sortable)
- UI: drop dual_llm option, keep only trusted/blocked

* fix: address greptile review feedback

- Remove redundant @@index([tool_name]) from schema.prisma (tool_name has @unique which already creates an index)
- Replace gen_random_uuid()::text with str(uuid.uuid4()) for portability
- Rewrite test_tool_registry_writer.py to mock execute_raw/query_raw (actual implementation) instead of Prisma model methods
- Fix test patches in test_tool_management_endpoints.py to target source modules since imports are inside function bodies
- Add "Tool Policies" page title to ToolPolicies.tsx

* fix: address greptile review round 2

- Replace NOW() with Python datetime parameter in tool_registry_writer (SQLite portability)
- Fix cache key collision in tool_policy_guardrail: use null-byte separator instead of colon
- Remove type==function filter from request-side tool extraction to match response-side behavior
- Clear seen_tool_names on flush so call_count increments per batch cycle not per pod lifetime

* fix: address greptile review round 3

- Fix test_seen_names_persist_across_flushes to match actual per-flush-cycle behavior
- Update module docstring in tool_discovery_queue.py to accurately describe flush behavior
- Add created_at/updated_at to raw SQL INSERT in batch_upsert_tools and update_tool_policy

* fix: cache tool policies per tool name not per combination

Previously the cache key was built from the full set of tool names in a
request, so each unique combination of tools got its own cold cache entry
and triggered a separate DB query. With N distinct tools across requests
this was effectively a DB hit on every request.

Now each tool name is cached individually. Cache hits are checked per
tool, only missing tools are fetched from DB in a single batch query,
and each result is cached separately. Once a tool's policy is warm,
any subsequent request using that tool benefits from the cache regardless
of what other tools are in the request.

* Update ui/litellm-dashboard/src/components/ToolPolicies.tsx

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-03 18:14:57 +05:30
yuneng-jiang
f13a8b6003 address greptile review feedback (greploop iteration 2)
- Wait for strategy select (API data loaded) before clicking Save
- Assert specific payload content in setCallbacksCall
- Move NotificationsManager import to top of file
2026-03-03 18:14:54 +05:30
yuneng-jiang
1fb5c496c8 address greptile review feedback (greploop iteration 1)
- Replace document.querySelector/querySelectorAll with screen.getByRole
- Replace raw dispatchEvent with userEvent.selectOptions
2026-03-03 18:14:54 +05:30
yuneng-jiang
0e8b5616d6 [Test] Add unit tests for router_settings components
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 18:14:54 +05:30
yuneng-jiang
86a884367f [Fix] UI - Virtual Keys: restrict Edit Settings button to key owners
Non-owner Internal Users could see and interact with the "Edit Settings"
button in the key Settings tab for keys they don't own. The button was
gated by `rolesWithWriteAccess.includes(userRole)` (role-only check)
instead of `canModifyKey` (ownership-aware), unlike the Regenerate and
Delete buttons which already used the correct check.

Replace the condition with `canModifyKey` so the Edit Settings button
follows the same proxy-admin / team-admin / key-owner logic as the
other action buttons. Add tests covering all permission paths.
2026-03-03 18:14:40 +05:30
Sameer Kankute
44e8971695 Fix: Transport Type for OpenAPI Spec on UI 2026-03-03 18:14:40 +05:30
Sameer Kankute
af2de6b366 fix: Missing OAuth session state. Please retry 2026-03-03 18:14:40 +05:30
Sameer Kankute
19a970f0eb Add Additonal header field on UI for testing passthrough 2026-03-03 18:14:40 +05:30
Harshit28j
b7f0721c66 security: fix critical/high CVEs in OS-level libs and NPM transitive 2026-03-03 18:13:36 +05:30
yuneng-jiang
36f7722b0f fix: add QueryClientProvider, remove stale file, use should naming in tests
- Wrap onboarding page with QueryClientProvider to prevent runtime crash
  (mirrors the same pattern used in LoginPage)
- Stage deletion of stale litellm/ui/litellm-dashboard/src/app/onboarding/page.tsx
  committed at the wrong path
- Rename all 16 test names to start with "should" per AGENTS.md convention
2026-02-23 21:53:42 -08:00
yuneng-jiang
a3491490f9 refactor onboarding 2026-02-23 21:30:08 -08:00
yuneng-jiang
02a53989cf fix: narrow onSubmit values and use semantic loading assertion 2026-02-23 21:20:22 -08:00
yuneng-jiang
0873494270 feat: extract OnboardingFormBody component with tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-23 21:08:19 -08:00
yuneng-jiang
a347cf0a33 feat: extract OnboardingErrorView component with tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-23 21:06:17 -08:00
yuneng-jiang
a3d4a8752f feat: extract OnboardingLoadingView component with tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-23 21:04:48 -08:00
yuneng-jiang
10a3304e6a refactor: simplify onboarding page to use OnboardingForm component 2026-02-23 20:18:39 -08:00
yuneng-jiang
1f8d23ec5b fix: set cookie path and add error feedback on claim failure 2026-02-23 18:16:38 -08:00
yuneng-jiang
2bc6dc5cad fix: sync email field value into AntD Form after credentials load 2026-02-23 18:12:17 -08:00
yuneng-jiang
884904c6d1 feat: add OnboardingForm component with variant prop
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-23 18:08:31 -08:00
yuneng-jiang
ba0ed492c4 fix: improve type safety and defensive guard in useOnboarding hook
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-23 18:05:23 -08:00
yuneng-jiang
f4303f5e22 feat: add useOnboarding hook with query and mutation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-23 18:01:10 -08:00
yuneng-jiang
715e134a66 adding testing coverage + fixing flaky tests 2026-02-23 17:23:10 -08:00