scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-14 23:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 102
39980 commits 2357 branches 1407 tags 2.7 GiB
Commit graph

3715 commits

Author SHA1 Message Date
yuneng-jiang
715e134a66 adding testing coverage + fixing flaky tests 2026-02-23 17:23:10 -08:00
yuneng-jiang
9f3fc492c2 adding tests 2026-02-23 16:02:36 -08:00
Alejandro Tapia
a1f5450b40 manualSorting fix. Filtering fetching fix 2026-02-23 16:01:04 -08:00
yuneng-jiang
09cc3b8bbc ui changes 2026-02-23 15:54:07 -08:00
Alejandro Tapia
bc99e38e55 Fix: accessToken not included in queryKey 2026-02-23 15:44:51 -08:00
Alejandro Tapia
f44c36f980 Fix Unbounded pagination 2026-02-23 15:43:04 -08:00
Alejandro Tapia
6d98622923 filters working properly for team virtual keys 2026-02-23 15:21:49 -08:00
yuneng-jiang
08f4a27e32 Merge remote-tracking branch 'origin' into litellm_blog_dropdown 2026-02-23 15:02:07 -08:00
Alejandro Tapia
34d09a314c virtual-keys-team-table 2026-02-23 14:47:07 -08:00
yuneng-jiang
1ecfbad46e adjust blog posts to fetch from github first 2026-02-23 14:45:05 -08:00
Krish Dholakia
4dfec4661b
Guardrail Monitor - measure guardrail reliability in prod (#21944) 
* fix: fix log viewer for guardrail monitoring

* feat(ui/): fix rendering logs per guardrail

* fix: fix viewing logs on overview tab of guardrail

* fix: log viewer

* fix: fix naming to align with metric
 2026-02-23 10:23:14 -08:00
Sameer Kankute
9b5bbee906
Merge pull request #21786 from BerriAI/litellm_oss_staging_02_21_2026 
Litellm oss staging 02 21 2026
 2026-02-23 18:51:55 +05:30
Sameer Kankute
37d45139f2
Merge pull request #21917 from BerriAI/litellm_fix_model_cost_map_wildcard 
Fix: Anthropic model wildcard access issue
 2026-02-23 18:45:49 +05:30
Sameer Kankute
4934d89cc7
Merge pull request #21872 from BerriAI/litellm_dev_02_21_2026_p4 
Litellm dev 02 19 2026 p2 (#21871)
 2026-02-23 18:35:57 +05:30
Sameer Kankute
c7aafdf794
Merge pull request #21926 from BerriAI/main 
merge main in oss 21 02
 2026-02-23 18:17:30 +05:30
Sameer Kankute
262235f266 Add ability to see which model cost map is getting used 2026-02-23 17:15:06 +05:30
Krish Dholakia
76ccc9e844
Guardrail Policy Versioning (#21862) 
* feat: initial commit, adding support for policy versioning on litellm

* fix(policy_registry): support policy versioning

* fix: multiple QA fixes for policy flow builder with guardrail versioning on litellm

* feat: ui improvements

* feat: add prisma migration

* fix: address greptile fixes
 2026-02-21 20:14:31 -08:00
Krrish Dholakia
9828b9929f fix(guardrails/usage_endpoints.py): fix usage dashboard 2026-02-21 19:33:50 -08:00
Krish Dholakia
886f1a3472
Litellm dev 02 19 2026 p2 (#21871) 
* feat(ui/): new guardrails monitor 'demo

mock representation of what guardrails monitor looks like

* fix: ui updates

* style(ui/): fix styling

* feat: enable running ai monitor on individual guardrails

* feat: add backend logic for guardrail monitoring
 2026-02-21 19:14:04 -08:00
yuneng-jiang
a0965d5b4a feat: add BlogDropdown to navbar after Docs link 2026-02-21 17:34:08 -08:00
yuneng-jiang
929d592ef4 fix: add retry, stable post key, enabled guard in BlogDropdown 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-02-21 17:34:08 -08:00
yuneng-jiang
1c0cfd7e3b fix: enforce 5-post limit in BlogDropdown and add cap test 2026-02-21 17:34:08 -08:00
yuneng-jiang
e241e6fd45 feat: add BlogDropdown component with react-query and error/retry state 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-02-21 17:34:08 -08:00
yuneng-jiang
98da524a9f fix: use existing useUISettings hook in useDisableShowBlog to avoid cache duplication 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-02-21 17:34:08 -08:00
yuneng-jiang
a82c8c18b2 fix: rename useUISettings to useUISettingsFlags to avoid naming collision 2026-02-21 17:34:08 -08:00
yuneng-jiang
70f8e979a0 feat: add useUISettings and useDisableShowBlog hooks 2026-02-21 17:34:08 -08:00
Ishaan Jaffer
67a64266ef fix rendering 2026-02-21 17:15:18 -08:00
milan-berri
5390469c1b
fix: replace Zapier webhook with Google Form for survey submission (#21621) 
* Replace Zapier webhook with Google Form for survey submission

* Add back error logging for survey submission debugging

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
 2026-02-21 16:55:59 -08:00
Ishaan Jaffer
7e2d6f2355 ui fix 2026-02-21 16:05:03 -08:00
Krish Dholakia
1f7eeb274c
Agent Builder - improve rejected response detection based on agent response (#21850) 
* fix: feat: add litellm_system_prompt support

* feat: support new 'litellm_agent' model provider

* feat: ui/ - new agent builder ui

* fix(anthropic/chat/transformation.py): normalize max_tokens if decimal

* feat(agentbuilderview.tsx): run compliance datasets against litellm agent

* feat: new response rejection detector

* fix: multiple fixes

* feat: add mcp tools support to agent builder

create an agent with access to llm's + mcp servers
 2026-02-21 15:34:42 -08:00
Krish Dholakia
9fc6fd647c
Agent Builder - support new experimental agent builder, to ensure agents pass compliance checks (#21817) 
* fix: feat: add litellm_system_prompt support

* feat: support new 'litellm_agent' model provider

* feat: ui/ - new agent builder ui

* fix(anthropic/chat/transformation.py): normalize max_tokens if decimal

* feat(agentbuilderview.tsx): run compliance datasets against litellm agent
 2026-02-21 15:32:47 -08:00
yuneng-jiang
3476452774
Merge pull request #21822 from dkindlund/fix/admin-ui-logging-metadata 
fix(ui): preserve logging_settings in key metadata on update
 2026-02-21 14:46:41 -08:00
Ishaan Jaff
235a47c576
fix(tests): mock test_claude_tool_use_with_gemini to fix flaky CI (#21832) 
* ui fixes

* fix(tests): mock test_claude_tool_use_with_gemini to avoid MALFORMED_FUNCTION_CALL flakiness
 2026-02-21 14:34:54 -08:00
Darien Kindlund
5f0bef3133 fix(ui): preserve logging_settings in key metadata on update 
The logging_settings condition used a bare truthiness check which
failed when the form field was undefined or not properly synced
from the EditLoggingSettings component. Changed to explicit
Array.isArray() check consistent with the tags field pattern.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-02-21 16:51:56 -05:00
shin-bot-litellm
1be30f5129
feat(router): Add complexity-based auto routing strategy (#21789) 
* feat(router): Add complexity-based auto routing strategy

Adds a rule-based routing strategy that classifies requests by complexity
and routes them to appropriate models - with zero API calls and sub-millisecond
latency.

## Features

- **Zero external API calls** - all scoring is local
- **Sub-millisecond latency** - typically <1ms per classification
- **Weighted multi-dimensional scoring** across 7 dimensions:
  - Token count (short=simple, long=complex)
  - Code presence (code keywords → complex)
  - Reasoning markers ("step by step" → reasoning tier)
  - Technical terms (domain complexity)
  - Simple indicators ("what is" → simple, negative weight)
  - Multi-step patterns (numbered steps)
  - Question complexity (multiple questions)
- **Configurable tier boundaries** and model mappings
- **Reasoning override** - 2+ reasoning markers force REASONING tier

## Usage

```yaml
model_list:
  - model_name: smart-router
    litellm_params:
      model: auto_router/complexity_router
      complexity_router_config:
        tiers:
          SIMPLE: gpt-4o-mini
          MEDIUM: gpt-4o
          COMPLEX: claude-sonnet-4
          REASONING: o1-preview
```

Inspired by ClawRouter: https://github.com/BlockRunAI/ClawRouter

## Files Added

- litellm/router_strategy/complexity_router/complexity_router.py - Main router class
- litellm/router_strategy/complexity_router/config.py - Configuration and defaults
- litellm/router_strategy/complexity_router/__init__.py - Package exports
- litellm/router_strategy/complexity_router/README.md - Documentation
- tests/test_litellm/router_strategy/test_complexity_router.py - Test suite (37 tests)

## Files Modified

- litellm/router.py - Integration with pre_routing_hook
- litellm/types/router.py - New config params

* feat(router): Add complexity-based auto routing strategy

Adds a new rule-based routing strategy that classifies requests by complexity
and routes them to appropriate models - without any external API calls.

## Features
- Weighted scoring across 7 dimensions: token count, code presence, reasoning
  markers, technical terms, simple indicators, multi-step patterns, questions
- Maps to 4 tiers: SIMPLE, MEDIUM, COMPLEX, REASONING
- Each tier configurable to a different model
- Zero API calls, <1ms latency
- Inspired by ClawRouter

## Configuration
```yaml
model_list:
  - model_name: smart_router
    litellm_params:
      model: auto_router/complexity_router
      complexity_router_config:
        tiers:
          SIMPLE: gemini-2.0-flash
          MEDIUM: gpt-4o-mini
          COMPLEX: claude-sonnet-4
          REASONING: claude-opus-4
```

## Use Cases
- Cost optimization: route simple queries to cheaper models
- Quality optimization: route complex queries to capable models
- Zero configuration: works out of the box with sensible defaults

* feat(router): Add complexity-based auto routing strategy

Adds a new rule-based routing strategy that classifies requests by complexity
and routes them to appropriate models - without any external API calls.

- Weighted scoring across 7 dimensions: token count, code presence, reasoning
  markers, technical terms, simple indicators, multi-step patterns, questions
- Maps to 4 tiers: SIMPLE, MEDIUM, COMPLEX, REASONING
- Each tier configurable to a different model
- Zero API calls, <1ms latency
- Inspired by ClawRouter

```yaml
model_list:
  - model_name: smart_router
    litellm_params:
      model: auto_router/complexity_router
      complexity_router_config:
        tiers:
          SIMPLE: gemini-2.0-flash
          MEDIUM: gpt-4o-mini
          COMPLEX: claude-sonnet-4
          REASONING: claude-opus-4
```

- Cost optimization: route simple queries to cheaper models
- Quality optimization: route complex queries to capable models
- Zero configuration: works out of the box with sensible defaults

* feat: add enterprise presets for complexity router

Adds preset configurations for different cloud providers:
- bedrock: AWS Bedrock (Claude models)
- vertex: Google Vertex AI (Gemini models)
- azure: Azure OpenAI (GPT + o1)
- standard: Direct API (OpenAI + Anthropic)
- cost_optimized: Maximum savings (Gemini Flash + cheaper models)

Usage:
```yaml
complexity_router_config:
  preset: bedrock  # or vertex, azure, standard, cost_optimized
```

* feat(ui): update auto router submit handler for complexity router

- Handle complexity_router model type in submit handler
- Generate correct litellm_params for complexity router:
  - model: auto_router/complexity_router
  - complexity_router_config: { tiers: { SIMPLE, MEDIUM, COMPLEX, REASONING } }
- Keep existing semantic router handling intact
- Add success notification with router type name

* docs: update PR description with UI changes

* chore: remove preset feature, keep simple tier config

* fix: exclude complexity_router from auto_router check

The _is_auto_router_deployment() was matching all auto_router/* models,
causing complexity_router to fail initialization. Now it explicitly
excludes auto_router/complexity_router which has its own handler.

* fix(complexity_router): Address Greptile review feedback

Fixes 5 issues flagged in code review:

1. **Mutable singleton mutation bug** - Now always creates a new
   ComplexityRouterConfig instance instead of reusing DEFAULT_COMPLEXITY_CONFIG
   singleton, preventing cross-instance config pollution.

2. **Substring matching false positives** - Added word boundaries (spaces)
   to short keywords like 'ok', 'try', 'api', 'git', 'node', 'java', 'vue'
   to prevent matching within longer words (e.g., 'capital' matching 'api').

3. **Redundant message extraction** - Simplified to single reverse loop that
   extracts both last user message and last system prompt efficiently.

4. **Unused imports** - Removed unused DEFAULT_CREATIVE_KEYWORDS and
   DEFAULT_MULTI_STEP_PATTERNS imports.

5. **Missing async_pre_routing_hook tests** - Added comprehensive tests for:
   - Multi-turn conversations
   - List-type content handling
   - No user message case
   - Empty string content
   - Message preservation
   - Singleton mutation prevention

* fix(complexity_router): Address Greptile review feedback

- Use word boundary matching for short keywords (<5 chars) to avoid
  false positives (e.g., 'api' matching 'capital', 'git' matching 'digital')
- Remove 'ok' from simple keywords (too many false positives)
- Add tests for keyword false positive prevention
- Fix test expectations for edge cases (empty string content, list content)

Addresses: 2/5 Greptile score feedback on PR #21789

* docs(auto_routing): Add complexity router documentation

- Add Complexity Router section to auto_routing.md
- Include comparison table with semantic auto router
- Add Python SDK and Proxy Server configuration examples
- Document all configuration options (tier boundaries, token thresholds, dimension weights)
- Explain how complexity scoring works

* feat(complexity_router): Add eval suite + tune scoring parameters

Added comprehensive evaluation suite with 29 test cases covering:
- SIMPLE tier: greetings, definitions, factual questions
- MEDIUM tier: technical explanations, comparisons, debugging
- COMPLEX tier: architecture design, complex coding
- REASONING tier: explicit reasoning requests
- Regression tests: substring false positive prevention

Tuned scoring parameters based on eval results:
- Lowered tier boundaries (0.15/0.35/0.60) for better tier distribution
- Increased code/technical weights (0.30/0.25) for complex prompts
- Reduced simple indicator weight (0.05) to avoid over-penalizing
- Fixed 'hey'/'hi' keywords to require leading space

Eval results: 29/29 passed (100%)

* fix(complexity_router): Address Greptile review round 2

1. **Empty user message handling** - Changed from falsy check to None check
   to properly distinguish 'no user message' from 'empty string message'

2. **ReDoS prevention** - Changed 'first.*then' to 'first.*?then' (non-greedy)
   to prevent regex backtracking on pathological inputs

3. **Documentation sync** - Updated README.md to match actual config values:
   - Tier boundaries: 0.15/0.35/0.60 (not 0.25/0.50/0.75)
   - Dimension weights: tokenCount=0.10, codePresence=0.30, technicalTerms=0.25,
     simpleIndicators=0.05, multiStepPatterns=0.03, questionComplexity=0.02

4. **Missing UI component** - Added ComplexityRouterConfig.tsx with:
   - Tier-to-model dropdown selectors
   - Descriptions and examples for each tier
   - How classification works explanation

5. **Inline import comment** - Added explanation for why ComplexityRouter
   import is inline (matches AutoRouter pattern, avoids circular imports)

* docs(auto_routing): fix dimension weights and tier boundaries to match config.py defaults

* fix(complexity_router): skip empty string content in async_pre_routing_hook

* fix(router): remove or {} masking None complexity_router_config

* fix(config): remove unused DEFAULT_MULTI_STEP_PATTERNS and DEFAULT_CREATIVE_KEYWORDS exports

* fix(complexity_router): use word boundary matching for all single-word keywords, avoid double-scanning reasoning keywords

* fix(router): clarify circular import comment for ComplexityRouter

* docs(README): fix token thresholds to match config.py defaults

* test(complexity_router): add false positive tests for error/class/merge keyword matching

* fix(complexity_router): align .get() fallbacks with config.py defaults, document system prompt scoring

* fix(config): deduplicate keywords across code and technical lists

---------

Co-authored-by: OpenClaw Assistant <assistant@openclaw.ai>
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
 2026-02-21 13:23:37 -08:00
Ishaan Jaff
349516280f
fix(ui): fix failing ui_unit_tests (#21792) 
- TeamMemberTab: pass canEditTeam=true so Add Member button renders
- UsagePageView: remove stale banner assertions (org/customer/agent banners were removed from source)
- LogDetailContent: scope '-' query to Provider description item using within() to avoid multiple-match error
 2026-02-21 11:33:28 -08:00
shin-bot-litellm
32b09b29fc
feat(ui): add forward_client_headers_to_llm_api toggle to general settings (#21776) 
* feat(ui): add forward_client_headers_to_llm_api toggle to general settings

* feat(ui): add forward_client_headers_to_llm_api toggle to UI Settings tab

- Add toggle to UISettings.tsx frontend (switch + label + description)
- Add field to UISettings model and ALLOWED_UI_SETTINGS_FIELDS
- Sync setting to general_settings on get/update so proxy picks it up at runtime

---------

Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
 2026-02-21 10:38:40 -08:00
yuneng-jiang
8ebaeb7229 fixing build 2026-02-21 09:13:33 -08:00
Zhenting Huang
647ea3bfcb
feat(proxy): add custom favicon support\n\nAdd ability to configure a custom favicon for the litellm proxy UI.\n\n- Add favicon_url field to UIThemeConfig model\n- Add LITELLM_FAVICON_URL env var support\n- Add /get_favicon endpoint to serve custom favicons\n- Update ThemeContext to dynamically set favicon\n- Add favicon URL input to UI theme settings page\n- Add comprehensive tests\n\nCloses #8323 (#21653) 2026-02-21 00:46:51 -08:00
janfrederickk
107f9dc46c
feat: show proxy url in ModelHub (#21660) 2026-02-20 22:26:14 -08:00
yuneng-jiang
1fb7320b6d
Merge pull request #21745 from BerriAI/litellm_org_member_email_ui 
[Feature] UI - Organization Info: Show member email, AntD tabs, reusable MemberTable
 2026-02-20 21:49:27 -08:00
yuneng-jiang
4597d34344 feat: org info page - AntD tabs, MemberTable with user_email 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-02-20 21:41:53 -08:00
yuneng-jiang
a5ee421eef refactor: TeamMemberTab uses shared MemberTable component 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-02-20 21:41:53 -08:00
yuneng-jiang
14e8af94a9 feat: add reusable MemberTable AntD component 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-02-20 21:41:53 -08:00
yuneng-jiang
9f3e20510d feat(usage): prefix credential tags and update Usage page banners 
- Prefix credential name tags with "Credential: " to distinguish them
  from user-defined tags when litellm_credential_name is injected
- Remove stale "new feature" banners from Organization, Customer, and
  A2A usage views
- Add closable info banner to Tag usage view noting that reusable
  credentials are automatically tracked and appear as "Credential: <name>"
 2026-02-20 20:55:57 -08:00
Ishaan Jaff
8e2aeeae0b
feat(ui): Guardrail Garden - guardrail marketplace (#21732) 
* feat(ui): add Guardrail Garden page with Vertex-style card layout

* feat(ui): add guardrail garden preset configs for form pre-fill

* feat(ui): add Guardrail Garden as first tab on guardrails page

* feat(ui): support preset prop in AddGuardrailForm for garden pre-fill

* fix(ui): fix merge syntax error in ComplianceUI

* refactor(ui): split guardrail_garden.tsx into focused modules
 2026-02-20 19:47:04 -08:00
Krish Dholakia
0888e17272
fix: ui fixes (#21731) 2026-02-20 19:29:55 -08:00
Ishaan Jaff
08520a9ed7
feat: add insults content filter + topic blocking compliance UI (#21729) 
* add denied_insults.yaml content filter category

* add block_insults.csv eval set (299 cases)

* add block_insults.jsonl eval set (299 cases)

* add insults eval results (100% F1)

* add TestInsultsContentFilter eval class

* add generate_compliance_prompts.py script

* add insultsCompliancePrompts.ts (299 prompts from CSV)

* add financialCompliancePrompts.ts (207 prompts from CSV)

* add Topic Blocking framework to compliance playground UI
 2026-02-20 19:10:31 -08:00
Krish Dholakia
e8d0afd7cb
Guardrail - competitor name blocker (#21719) 
* feat: add competitor name blocker guardrail

* fix: fix batch test endpoint for compliance playground

* fix(airline.py): add list of all known airlines to airline competitor name detector

prevent competitor discussion on company chatbot

* feat: ui tweaks for prod
 2026-02-20 18:52:40 -08:00
Ishaan Jaff
c61dea5af9
UI: Redesign guardrail creation form with vertical stepper (#21727) 
* ui: redesign guardrail creation form with inline vertical stepper

Replace horizontal Ant Design Steps with an inline vertical stepper.
Completed steps collapse to a single line, active step expands.
Switch to Tremor buttons, rename steps for clarity.

* ui: rename Content Categories to Blocked topics and fix overflow

Update heading and description text, add flexWrap to prevent
text from going off-screen, fix YAML preview overflow with
pre-wrap and word-break.

* feat: support explicit display_name in content filter category YAML

Check for a display_name field before auto-generating from
category_name. Lets categories have human-friendly names
without changing their API identifier.

* fix: update denied_financial_advice display name

Add display_name field so it shows as
"Denied Financial / Investment Advice" in the UI.
 2026-02-20 18:42:11 -08:00