Commit graph

10 commits

Author SHA1 Message Date
ryan-crabbe-berri
27069bd74f
feat(ui): shadcn migration foundation: Tailwind v4, shadcn init, antd cascade fix (#31995)
* feat(ui): shadcn migration foundation: Tailwind v4, shadcn init, antd cascade fix

Upgrade the dashboard from Tailwind v3 to v4 with CSS-first config: the
official upgrade codemod renamed utilities across 151 files, and
tailwind.config.js (plus the dead tailwind.config.ts) is replaced by
@theme tokens, @source globs, and @plugin directives in globals.css. The
Tremor safelist becomes @source inline patterns and the legacy tremor
theme tokens carry over verbatim. ui_colors.json was build-time only and
fed the dying Tremor palette, so its brand values are inlined and the
file removed; runtime theming replaces that path next.

shadcn is initialized with a hand-authored components.json (rsc,
cssVariables, baseColor gray) pointing utils at the existing
lib/cva.config.ts, which now exports cn (cva beta cx + twMerge) instead
of adding class-variance-authority as a second variant library. The two
ad-hoc cn helpers fold into it. Button lands as the canary primitive,
adapted to cva beta and React 18 forwardRef, with tests covering the
variant, twMerge, asChild, and ref seams. --radius is 0.5rem so the
shadcn radius scale reproduces Tailwind defaults and legacy rounded-*
classes render unchanged.

antd v5 emits unlayered CSS-in-JS that would beat every layered v4
utility, so AntdGlobalProvider now wraps the app in StyleProvider layer
and ConfigProvider cssVar, and globals.css declares
@layer theme, base, antd, components, utilities. antd wins over
preflight but yields to utilities, which is what lets migrated shadcn
pages coexist with legacy antd pages. Preflight stays global with the
three v3 behaviors pinned (default border color, button cursor,
placeholder color).

* fix(ui): restore tremor opacity tints removed by tailwind v4

Tailwind v4 removed the *-opacity-* utilities, but the precompiled
@tremor/react dist still composes them with shade-500 palette classes
(bg-opacity-10 over bg-<color>-500 etc.), so Badge, BadgeDelta, Callout,
light Icon and Button, BarList, and ProgressBar lost their tints and
rendered solid 500-shade fills. Adversarial review caught it; the
original smoke pages only exercised antd Tags.

tremor-v3-compat.css restores exactly the pairs tremor emits: for each
of the 22 safelisted colors, bg-opacity-{10,20,40}, hover/group-hover
bg-opacity-{20,30}, and ring-opacity-{20,40} against the -500 shade,
via color-mix into the utilities layer. Tremor's colorPalette maps both
background and iconRing to 500, so the -500 pairing covers every
composition in the dist; dark: variants are inert until dark mode ships.
The shim dies with @tremor/react at the end of the migration.

The upgrade codemod also missed two hand-rolled modal scrims using
bg-black bg-opacity-{30,50} (solid black under v4); now bg-black/30 and
bg-black/50. Removed the docker/build_admin_ui.sh copy of
enterprise_colors.json into the deleted ui_colors.json; that build-time
rebrand path is retired and its runtime replacement lands with the
theming phase.

* fix(ui): pair ring-opacity-40 with shade 300 in tremor compat shim

Tremor's colorPalette maps ring to shade 300, and the only consumer of
ring-opacity-40 (Icon variant outlined) composes it with that shade,
so the shade-500 rows were dead and outlined icon rings would render
at full opacity. Latent today (no dashboard usage of the outlined
variant); caught by adversarial review. ring-opacity-20 stays at 500
(iconRing), matching Badge and BadgeDelta.
2026-07-02 19:02:27 -07:00
ryan-crabbe-berri
7edf3a9cb5
style(ui): run prettier --write across the dashboard (#29622)
Formatting-only pass; no logic changes. Brings the UI into compliance
with .prettierrc so the new format-check CI job passes
2026-06-04 11:37:54 -07:00
Krish Dholakia
ff8e01d20b
feat: Add Canadian PII protection (PIPEDA) (#22951)
* feat: Add Canadian PII protection patterns and PIPEDA-compliant policy template

Adds 6 new Canadian PII regex pattern detectors to patterns.json:
- ca_sin: Social Insurance Number (PIPEDA Privacy Act, Income Tax Act)
- ca_ohip: Ontario Health Insurance Plan Number (PHIPA)
- ca_on_drivers_licence: Ontario driver's licence (HTA, PIPEDA)
- ca_immigration_doc: IRCC immigration docs (UCI, work/study permits, IMM refs)
- ca_bank_account: Canadian bank account routing (transit-institution-account)
- ca_postal_code: Canadian postal code (Canada Post spec)

Adds comprehensive policy template 'canadian-pii-protection' (id: canadian-pii-protection)
with 5 sub-guardrails grouping patterns by data type. All patterns include contextual
keyword matching (English + French keywords where applicable) to reduce false positives.
Complements existing passport_canada pattern.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

* feat: Add Canadian PII compliance dataset and tests (57 tests)

Adds:
- test_ca_patterns.py: 30 unit tests for regex pattern matching (SIN, OHIP,
  driver's licence, immigration docs, bank account, postal code)
- test_ca_policy_e2e.py: 27 end-to-end tests running the full
  ContentFilterGuardrail pipeline with MASK action — validates detection
  of real PII and pass-through of clean prompts
- canadianPiiCompliancePrompts.ts: 21-prompt compliance dataset for UI
  evaluation, wired into the main compliancePrompts framework

Fixes keyword_pattern alternation ordering in patterns.json — longer
alternatives (e.g. "social insurance number") now precede shorter ones
("social insurance") to avoid excessive gap-word count when the regex
engine selects the shorter match first.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

* feat: Add University of Toronto FIPPA identifier patterns and tests (36 tests)

Add 3 UofT institutional identifiers (student/employee number, UTORid, TCard)
covered under Ontario FIPPA. Includes pattern definitions, policy template
sub-guardrail, compliance prompts, unit tests, and e2e tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: Correct test assertion bug and inaccurate docstring

Fix test_utorid_masked checking `result` (dict) instead of `output` (string).
Update test_ca_policy_e2e.py docstring to clarify scope vs UofT tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: Address Greptile review feedback

- Tighten ca_postal_code keyword_pattern: replace broad "address" with
  specific compound terms (mailing/street/shipping/home address)
- Add missing "PIPEDA" tag to policy_templates.json for discoverability
- Add us_phone pattern to test_ca_policy_e2e.py setup to match deployed template
- Add phone number e2e test for complete coverage

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: Tighten patterns to reduce false positives and add missing test coverage

- ca_sin: reject leading-zero SINs ([1-9]\d{2}), set allow_word_numbers to false
- ca_immigration_doc: require separators in UCI pattern (prevent bare \d{10} match)
- uoft_utorid: qualify generic keywords (acorn -> acorn login, quercus -> quercus login)
- uoft_tcard: remove generic keywords (student card, id card, library card) that
  overlap with credit card contexts; keep only UofT-specific terms (tcard, campus card)
- Add visa/mastercard/amex/iban patterns to test_ca_policy_e2e.py setup to match
  deployed template; add Visa card masking test
- Add test verifying "student card" no longer triggers TCard redaction

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-06 18:27:31 -08:00
Krish Dholakia
c2c8870d2d
Add claims agent guardrails (5 categories + policy template) (#22113)
* Add claims agent guardrails with 243-case eval dataset

5 new category guardrails for healthcare claims agent chatbots:
- claims_fraud_coaching: fraud coaching, exaggeration, document forgery
- claims_phi_disclosure: unauthorized PHI access, bulk data extraction
- claims_prior_auth_gaming: code manipulation, medical necessity misrepresentation
- claims_system_override: system injection, rule bypass, role impersonation
- claims_medical_advice: medical advice (claims-context-aware)

Plus claims_agent_safety.yaml policy template combining all 5.

All 5 eval suites pass at 100% precision/recall/F1 (243 test cases).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add claims agent chatbot safety policy template

Combines the 5 claims guardrails into a single deployable policy template:
fraud coaching, PHI disclosure, prior-auth gaming, system override, and medical advice.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add guardrail benchmark results and UI compliance prompts

Adds benchmark results for claims, discrimination, and content filter guardrails.
Updates UI compliance prompt data.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove duplicate "file an appeal" exception in claims_prior_auth_gaming.yaml

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove unused claims_agent_safety.yaml policy template

The claims-agent-safety template in policy_templates.json references
individual category files in categories/, not this combined file.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 18:09:38 -08:00
Krish Dholakia
12c4876891
Agents - assign tools (#22064)
* feat(proxy): add max_iterations limiter for agent session loops (#22058)

Adds a new proxy hook that enforces a per-session cap on the number of
LLM calls an agentic loop can make. Callers send a session_id with each
request, and the hook counts calls per session, returning 429 when the
configured max_iterations limit is exceeded.

- Uses Redis Lua script for atomic increment (multi-instance safe)
- Falls back to in-memory cache when Redis unavailable
- Follows parallel_request_limiter_v3 pattern
- Configurable via key metadata: {"max_iterations": 25}
- Session counters auto-expire via TTL (default 1hr)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add new code execution dataset

* feat(agent_endpoints/): allow giving agents keys

* fix: ui fixes

* feat: allow assigning mcp servers to agents

* fix: eliminate duplicate DB queries in MCP agent auth and N+1 in agent listing (#22110)

- Extract _get_agent_object_permission helper so _get_allowed_mcp_servers_for_agent
  and _get_agent_tool_permissions_for_server share a single DB fetch instead of
  each independently querying the same agent row (was 1+N queries per MCP request)
- Use include={"object_permission": True} on find_many in get_all_agents_from_db
  to eagerly load permissions in one query instead of N+1
- Use include={"object_permission": True} on create/update/find_unique in all
  agent CRUD operations, removing attach_object_permission_to_dict follow-up calls

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 11:44:30 -08:00
Krish Dholakia
00ab4d2067
feat: add new code execution dataset (#22065) 2026-02-24 20:58:23 -08:00
Krish Dholakia
76ccc9e844
Guardrail Policy Versioning (#21862)
* feat: initial commit, adding support for policy versioning on litellm

* fix(policy_registry): support policy versioning

* fix: multiple QA fixes for policy flow builder with guardrail versioning on litellm

* feat: ui improvements

* feat: add prisma migration

* fix: address greptile fixes
2026-02-21 20:14:31 -08:00
Ishaan Jaff
08520a9ed7
feat: add insults content filter + topic blocking compliance UI (#21729)
* add denied_insults.yaml content filter category

* add block_insults.csv eval set (299 cases)

* add block_insults.jsonl eval set (299 cases)

* add insults eval results (100% F1)

* add TestInsultsContentFilter eval class

* add generate_compliance_prompts.py script

* add insultsCompliancePrompts.ts (299 prompts from CSV)

* add financialCompliancePrompts.ts (207 prompts from CSV)

* add Topic Blocking framework to compliance playground UI
2026-02-20 19:10:31 -08:00
Krish Dholakia
e8d0afd7cb
Guardrail - competitor name blocker (#21719)
* feat: add competitor name blocker guardrail

* fix: fix batch test endpoint for compliance playground

* fix(airline.py): add list of all known airlines to airline competitor name detector

prevent competitor discussion on company chatbot

* feat: ui tweaks for prod
2026-02-20 18:52:40 -08:00
Krish Dholakia
a385126a7c Litellm dev compliance UI (#21457)
* feat(ui/): initial commit adding a compliance testing playground

allow proxy admins to test policies and guardrails against datasets

* feat(ui/): make score more friendly

* feat(policy_endpoints.py): new helper function for testing policies

* feat(policy_endpoints.py): expose new endpoint for testing policies and guardrails

enables compliance playground to work as expected

* feat(complianceui.tsx): show returned text
2026-02-18 18:22:43 -08:00