Commit graph

4830 commits

Author SHA1 Message Date
Ishaan Jaffer
e4752f4f9d ui fix 2026-02-17 18:02:33 -08:00
Ishaan Jaff
5a3a0210cb
feat(ui): add guardrail jump link in log detail view (#21437)
* feat(ui): add guardrail jump link at top of log detail

* fix(ui): align guardrail jump link to the left

* fix(ui): move guardrail jump link to trace sidebar

* fix(ui): move guardrail pill above event rows in sidebar
2026-02-17 17:33:06 -08:00
Ishaan Jaffer
f329294005 fix(ui): show category count badge in guardrail selection modal
Category-based guardrails (like EU AI Act) now display an orange
tag showing how many categories they contain, matching the existing
pattern count tag for pattern-based guardrails.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-17 16:57:31 -08:00
yuneng-jiang
ec19005942 refactor - change key type label 2026-02-16 21:08:13 -08:00
Ishaan Jaff
0f96b5a1b7
feat: guardrail tracing UI - policy, detection method, match details (#21349)
* feat: add GuardrailTracingDetail TypedDict and tracing fields to StandardLoggingGuardrailInformation

* feat: add policy_template field to Guardrail config TypedDict

* feat: accept GuardrailTracingDetail in base guardrail logging method

* feat: populate tracing fields in content filter guardrail

* test: add tracing fields tests for custom guardrail base class

* test: add tracing fields e2e tests for content filter guardrail

* feat: add guardrail tracing UI - policy badges, match details, timeline

* feat: redesign GuardrailViewer to Guardrails & Policy Compliance layout

Two-column layout with request lifecycle timeline on the left
and compact evaluation detail cards on the right. Header shows
guardrail count, pass/fail status, total overhead, policy info,
and an export button.

* feat: add clickable guardrail link in metrics + show policy names

* feat: add risk_score field to StandardLoggingGuardrailInformation

* feat: compute risk_score in content filter guardrail

* feat: display backend risk_score badge on evaluation cards

* fix: fallback to frontend risk score when backend doesn't provide one
2026-02-16 18:44:27 -08:00
yuneng-jiang
df5e8d01a6 address greptile review feedback (greploop iteration 2)
- Default user_id to caller's own ID for non-admins instead of 403 when
  omitted, preserving backward compatibility for API consumers
- Apply same fix to aggregated endpoint
- Update test to verify defaulting behavior instead of expecting 403
- Add useEffect to sync selectedUserId when auth state settles in
  UsagePageView to handle async auth initialization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-16 17:37:18 -08:00
yuneng-jiang
0d2aac6928 allow filtering by user in global usage 2026-02-16 16:46:09 -08:00
Ishaan Jaff
1a8525d02a
Add GDPR Art. 32 EU PII Protection Policy Template (#21340)
* Add 6 new EU PII patterns for GDPR compliance

- fr_nir: French Social Security Number (NIR/INSEE) with validation
- eu_iban_enhanced: Enhanced IBAN detection with specific format
- fr_phone: French phone numbers (+33, 0033, 0 formats)
- eu_vat: EU VAT identification numbers (all 27 member states)
- eu_passport_generic: Generic EU passport format
- fr_postal_code: French postal codes with contextual keywords

* Add GDPR Art. 32 EU PII Protection policy template

- Comprehensive GDPR Article 32 compliance policy
- 4 guardrail groups: National IDs, Financial, Contact Info, Business IDs
- Masks French NIR/INSEE, EU IBANs, French phones, EU VAT numbers
- Includes EU passport numbers and email addresses
- Medium complexity template with indigo icon

* Add comprehensive tests for EU PII patterns

- Test French NIR validation (sex digit, month range)
- Test enhanced IBAN detection (French, German)
- Test French phone number formats
- Test EU VAT numbers
- Test generic EU passport format
- Test French postal code pattern

* Add EU pattern loading and category validation tests

- Verify all 6 EU PII patterns are loaded correctly
- Verify patterns are categorized as 'EU PII Patterns'
- Ensure pattern loading consistency

* Add end-to-end tests for GDPR policy template

- 4 tests for PII that should be masked (NIR, IBAN, phone, VAT)
- 4 tests for text that should pass through (invalid patterns, no PII)
- 1 bonus test for multiple PII types in same message
- All tests verify correct masking behavior

* Add region field to policy templates

- Added region field to all 6 templates (EU, AU, Global)
- Updated both main and backup JSON files
- Enables region-based filtering in UI

* Add region filter to policy templates UI

- Added Radio.Group filter for regions (All, AU, EU, Global)
- Efficient filtering with useMemo hooks
- Clean button-based UI matching existing design
- Defaults missing regions to Global

* Apply suggestion from @greptile-apps[bot]

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Address Greptile review: add contextual guards and negative tests

- Added keyword_pattern to eu_vat (VAT, tax number, fiscal code, etc.)
- Added keyword_pattern to eu_passport_generic (passport, travel document, etc.)
- Added 3 negative unit tests for false positive prevention
- Added 2 E2E tests verifying no masking without keyword context
- All patterns now require contextual keywords to prevent false positives

* address greptile review feedback (greploop iteration 1)

- Remove unused HTTPException import from test file
- Add keyword_pattern to eu_vat for contextual VAT matching
- Add allow_word_numbers: false to eu_passport_generic
- Add negative test cases for EU VAT false positives
- All 5 Greptile comments addressed

* Address Greptile feedback: fix patterns and sync backup

- Fix fr_phone pattern: use negative lookbehind (?<!\d) to prevent false matches in longer digit strings
- Add keyword_pattern to eu_passport_generic to reduce false positives on version strings/SKUs
- Sync policy_templates_backup.json with main file (add GDPR template)
- Add keyword_pattern to eu_vat (was auto-added by formatter)

All pattern tests passing

* address greptile review feedback (greploop iteration 2)

- Update test to document that eu_vat raw pattern is intentionally broad
- Test verifies pattern DOES match common words (by design)
- Documents that keyword_pattern guard prevents false positives in production
- Addresses Greptile's false positive risk concern

* address greptile review feedback (greploop iteration 3)

- Fix test_eu_vat_masked: change gap from 2 words to 1 word (VAT number: FR...)
- This ensures keyword matching works within MAX_KEYWORD_VALUE_GAP_WORDS=1 limit
- fr_phone pattern already works correctly (verified with tests)
- test_pattern_requires_keyword_context already updated in iteration 2

Addresses final issues from Greptile 2/5 review

* fix: remove country-specific passport patterns from GDPR template

- Remove passport_france, passport_germany, passport_netherlands from template
- These patterns lack keyword guards and cause false positives
- Only eu_passport_generic remains (has keyword_pattern guard)
- Sync policy_templates_backup.json with main file
- Update test setup to match template

All 11 E2E tests now passing 

* Update tests/test_litellm/proxy/guardrails/guardrail_hooks/content_filter/test_gdpr_policy_e2e.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/content_filter.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-16 15:28:46 -08:00
yuneng-jiang
51716866cc
Merge pull request #21007 from atapia27/feat/playground-test-fallbacks
Feat/playground test fallbacks
2026-02-16 14:56:59 -08:00
Alejandro Tapia
c9e15005b1 differentiate solely by model_Id 2026-02-16 12:37:53 -08:00
Alejandro Tapia
a36d7dcb81 Tooltip removed and replaced with Popover (triggered by icon only), including url to router settings 2026-02-16 12:05:56 -08:00
Krish Dholakia
45690db820
UI - Content Filters, help edit/view categories and 1-click add categories + go to next page (#21223)
* feat(ui/): allow viewing content filter categories on guardrail info

* fix(add_guardrail_form.tsx): add validation check to prevent adding empty content filter guardrails

* feat(ui/): improve ux around adding new content filter categories

easy to skip adding a category, so make it a 1-click thing
2026-02-16 09:11:16 -08:00
Harshit Jain
56b53cb502
Merge branch 'main' into litellm_fix-virtual-key-grace-period 2026-02-15 08:09:13 +05:30
yuneng-jiang
ac648af78e add timeout to long running tests 2026-02-14 18:05:27 -08:00
yuneng-jiang
1c1807df45 defensive checks for null 2026-02-14 17:30:58 -08:00
yuneng-jiang
b3fd76a713 adding access groups on keys and teams 2026-02-14 17:01:29 -08:00
yuneng-jiang
27fe16be4b Merge remote-tracking branch 'origin/main' into litellm_ui_access_groups 2026-02-14 16:33:44 -08:00
yuneng-jiang
3e97c0d5d6 temp commit for merge 2026-02-14 16:33:35 -08:00
yuneng-jiang
55f169aa69
Merge pull request #21190 from BerriAI/litellm_access_groups_inte
[Feature] Access Group Checks
2026-02-14 16:28:09 -08:00
yuneng-jiang
0d6fcee23e injecting localstorage values to hide usageIndicator, prompts, and new badges 2026-02-14 15:49:27 -08:00
yuneng-jiang
5cf91e573e Merge remote-tracking branch 'origin' into litellm_access_groups_inte 2026-02-14 13:27:36 -08:00
Krish Dholakia
644f66a7d3
Agent Guardrails - on streaming output (#21206)
* fix(ui/): add mcp input as an example for custom code guardrails

* feat(a2a/): ensure a2a guardrails works on response output

* feat(a2a/): support streaming guardrails

* fix(ui/): add mcp input as an example for custom code guardrails
2026-02-14 11:36:52 -08:00
Ishaan Jaffer
977d1f90d2 test fixes 2026-02-14 11:33:45 -08:00
Krish Dholakia
e41ecc7a71
Guardrails - add nsfw policy template, toxic keywords in multiple languages, child safety content filter, json content viewer (#21205)
* fix(content_filter.py): fix filter on toxic keywords

* feat: improve toxic/abusive language detection

* fix: additional improvements to nsfw filters

* feat: more improvements to nsfw filter

* feat(content_filter.json): add new australia specific nsfw content filter

ensure complete coverage for australia nsfw

* fix: cleanup policy templates

* fix(index.tsx): alert notice

* fix(index.tsx): add disclaimer notice

* feat(harmful_child_safety.yaml): new child safety content filter

ensure we catch inappropriate, child-specific content

* feat(policy_templates.json): add child safety and self harm filters

* fix(content_filter.py): improve racial bias filter to use a similar identifier + block word pattern and cover a wider range of ethnicities

* feat(policy_templates.json): add racial bias to nsfw policy template

* feat: add json content viewer
2026-02-14 09:41:55 -08:00
yuneng-jiang
890927b69c Merge remote-tracking branch 'origin' into litellm_access_groups_inte 2026-02-14 09:06:49 -08:00
yuneng-jiang
91226fb0bb adjusting UI for model name 2026-02-13 22:07:24 -08:00
yuneng-jiang
5525dd4f20 access groups pt 2 2026-02-13 21:50:16 -08:00
Ishaan Jaff
96802e177b
Add pipeline flow builder UI for guardrail policies (#21188)
* Add pipeline type definitions for guardrail pipelines

PipelineStep, GuardrailPipeline, PipelineStepResult, PipelineExecutionResult
with validation for actions (allow/block/next/modify_response) and modes.

* Export pipeline types from policy_engine types package

* Add optional pipeline field to Policy model

* Add pipeline executor for sequential guardrail execution

* Parse pipeline config in policy registry

* Add pipeline validation in policy validator

* Add pipeline resolution and managed guardrail tracking

* Resolve pipelines and exclude managed guardrails in pre-call

* Integrate pipeline execution into proxy pre_call_hook

* Add test guardrails for pipeline E2E testing

* Add example pipeline config YAML

* Add unit tests for pipeline type definitions

* Add unit tests for pipeline executor

* Add pipeline column to LiteLLM_PolicyTable schema

* Add pipeline field to policy CRUD request/response types

* Add pipeline support to policy DB CRUD operations

* Add PipelineStep and GuardrailPipeline TypeScript types

* Add Zapier-style pipeline flow builder UI component

* Integrate pipeline flow builder with mode toggle in policy form

* Add pipeline display section to policy info view

* Add unit tests for pipeline in policy CRUD types

* Refactor policy form to show mode picker first with icon cards

* Add full-screen FlowBuilderPage component for pipeline editing

* Wire up full-screen flow builder in PoliciesPanel with edit routing

* Restyle flow builder to match dev-tool UI aesthetic

* Restyle flow builder cards to match reference design

* Update step card to expanded layout with stacked ON PASS / ON FAIL sections

* Add end card to flow builder showing return to normal control flow

* Add PipelineTestRequest type for test-pipeline endpoint

* Export PipelineTestRequest from policy_engine types

* Add POST /policies/test-pipeline endpoint

* Add testPipelineCall networking function

* Add PipelineStepResult and PipelineTestResult types

* Add test pipeline panel to flow builder with run button and results display

* Fix pipeline executor: inject guardrail name into metadata so should_run_guardrail allows execution

* Update litellm/proxy/policy_engine/pipeline_executor.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update litellm/proxy/utils.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update litellm/proxy/policy_engine/policy_endpoints.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update litellm/proxy/policy_engine/pipeline_executor.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-13 20:06:03 -08:00
shin-bot-litellm
933955bfa0
fix(ci): Fix E2E login button selector - use exact match (#21176)
* fix(ci): Fix ruff lint error - unused import

Remove unused 'cast' import in vertex_ai_ingestion.py (ruff F401)

* fix(ci): Fix E2E login button selector - use exact match

Login button selector now matches both 'Login' and 'Login with SSO',
causing strict mode violation. Use { exact: true } to match only 'Login'.

---------

Co-authored-by: OpenClaw <openclaw@users.noreply.github.com>
2026-02-13 18:20:10 -08:00
yuneng-jiang
7561c1a3c3 addressing comments 2026-02-13 16:54:34 -08:00
yuneng-jiang
ca2ce4a946 adding tests 2026-02-13 16:30:24 -08:00
yuneng-jiang
2f3dc4860f new badge changes 2026-02-13 16:22:07 -08:00
yuneng-jiang
5dbcca8d43 Access groups UI 2026-02-13 16:20:31 -08:00
Harshit Jain
f87b12a2f7
fix grace period with better error handling on frontend and as per best practices 2026-02-14 04:18:08 +05:30
yuneng-jiang
b65cb646aa addressing comments 2026-02-13 13:59:49 -08:00
yuneng-jiang
938a920372 cost breakdown fix 2026-02-13 12:51:55 -08:00
Ishaan Jaffer
817de99177 UI build 2026-02-13 12:51:17 -08:00
yuneng-jiang
71107b3fd0
Merge pull request #21149 from BerriAI/litellm_spendlogs_reset_custom_time
[Fix] UI - Spend Logs: Reset Filters Resets Custom Date Range
2026-02-13 12:30:45 -08:00
yuneng-jiang
eb144a44d0
Merge pull request #21144 from BerriAI/litellm_ui_fallbacks_10
[Refactor] UI - Fallbacks: Default Configurable to 10 Models
2026-02-13 12:30:35 -08:00
yuneng-jiang
82fcc39e6b reset custom time range when reset filters is clicked 2026-02-13 12:12:50 -08:00
yuneng-jiang
6eeb9b109b
Merge pull request #21143 from BerriAI/litellm_model_id_search
[Feature] UI - Spend Logs: Sorting Columns
2026-02-13 11:53:16 -08:00
yuneng-jiang
724ab1c154
Update ui/litellm-dashboard/src/components/networking.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-13 11:53:04 -08:00
Krish Dholakia
24b56a14eb
Guardrails - new Policy Templates (pre-configured guardrail combinations for specific use-cases) (#21025)
* feat(patterns.json): add australia specific pii patterns - tax file number, abn, medicare number

Improve PII detection for australian contexts

* feat(patterns.json): add iban + street address pattern detection

* feat: support policy templates on ui

allows admin to enable pre-configured guardrails

helps cover specific use-cases well

* feat: create missing guardrails, working policy templates

* feat: policy_templates.json

support hosted policy templates

allows others to contribute to the policy templates

* docs: document new policy templates

* fix: address greptile feedback

* fix: fix linting error
2026-02-13 11:53:02 -08:00
yuneng-jiang
25fccad127 Change default fallbacks to 10 models 2026-02-13 11:50:13 -08:00
yuneng-jiang
adc2859f0a sorting spend logs in ui 2026-02-13 11:33:41 -08:00
yuneng-jiang
5ddba48df9
Merge pull request #21088 from BerriAI/litellm_spend_rebase
[Fix] Spend Management Tests
2026-02-13 09:11:42 -08:00
Sameer Kankute
e17c639fb1
Merge pull request #21085 from BerriAI/litellm_oss_staging_02_13_2026
Litellm oss staging 02 13 2026
2026-02-13 18:38:14 +05:30
Harshit Jain
673b7d1fea
Merge branch 'main' into litellm_fix-virtual-key-grace-period 2026-02-13 18:14:47 +05:30
yuneng-jiang
9a2410be71
Merge pull request #20922 from atapia27/feat/fallback-display
fallback-display: updated fallback display table to use arrows and card structure for better visibility.
2026-02-12 22:05:32 -08:00
yuneng-jiang
21b38a4030 Merge pull request #20720 from OrionCodeDev/fix-spend-logs
fix-spend-logs
2026-02-12 21:37:00 -08:00