Commit graph

4746 commits

Author SHA1 Message Date
yuneng-jiang
d98440f452 Merge remote-tracking branch 'origin' into litellm_yj_march_18_2026 2026-03-18 16:20:07 -07:00
yuneng-jiang
83d185d867
Merge pull request #24039 from BerriAI/litellm_/vibrant-hypatia
[Fix] UI - Default Team Settings: Add Missing Permission Options
2026-03-18 15:57:51 -07:00
yuneng-jiang
1e6abf8142
Merge branch 'main' into litellm_yj_march_17_2026 2026-03-18 15:13:41 -07:00
Ishaan Jaff
8e61b32b8e
[Staging] - Ishaan March 17th (#23903)
* feat(xai): add grok-4.20 beta 2 models with pricing (#23900)

Add three grok-4.20 beta 2 model variants from xAI:
- grok-4.20-multi-agent-beta-0309 (reasoning + multi-agent)
- grok-4.20-beta-0309-reasoning (reasoning)
- grok-4.20-beta-0309-non-reasoning

Pricing (from https://docs.x.ai/docs/models):
- Input: $2.00/1M tokens ($0.20/1M cached)
- Output: $6.00/1M tokens
- Context: 2M tokens

All variants support vision, function calling, tool choice, and web search.
Closes LIT-2171

* docs: add Quick Install section for litellm --setup wizard (#23905)

* docs: add Quick Install section for litellm --setup wizard

* docs: clarify setup wizard is for local/beginner use

* feat(setup): interactive setup wizard + install.sh (#23644)

* feat(setup): add interactive setup wizard + install.sh

Adds `litellm --setup` — a Claude Code-style TUI onboarding wizard that
guides users through provider selection, API key entry, and proxy config
generation, then optionally starts the proxy immediately.

- litellm/setup_wizard.py: wizard with ASCII art, numbered provider menu
  (OpenAI, Anthropic, Azure, Gemini, Bedrock, Ollama), API key prompts,
  port/master-key config, and litellm_config.yaml generation
- litellm/proxy/proxy_cli.py: adds --setup flag that invokes the wizard
- scripts/install.sh: curl-installable script (detect OS/Python, pip
  install litellm[proxy], launch wizard)

Usage:
  curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/install.sh | sh
  litellm --setup

* fix(install.sh): remove orange color, add LITELLM_BRANCH env var for branch installs

* fix(install.sh): install from git branch so --setup is available for QA

* fix(install.sh): remove stale LITELLM_BRANCH reference that caused unbound variable error

* fix(install.sh): force-reinstall from git to bypass cached PyPI version

* fix(install.sh): show pip progress bar during install

* fix(install.sh): always launch wizard via $PYTHON_BIN -m litellm, not PATH binary

* fix(install.sh): use litellm.proxy.proxy_cli module (no __main__.py exists)

* fix(install.sh): suppress RuntimeWarning from module invocation

* fix(install.sh): use Python bin-dir litellm binary to avoid CWD sys.path shadowing

* fix(install.sh): use sysconfig.get_path('scripts') to find pip-installed litellm binary

* fix(install.sh): redirect stdin from /dev/tty on exec so wizard gets terminal, not exhausted pipe

* fix(install.sh): warn about git clone duration, drop --no-cache-dir so re-runs are faster

* feat(setup_wizard): arrow-key selector, updated model names

* fix(setup_wizard): use sysconfig binary to start proxy, not python -m litellm

* feat(setup_wizard): credential validation after key entry + clear next-steps after proxy start

* style(install.sh): show git clone warning in blue

* refactor(setup_wizard): class with static methods, use check_valid_key from litellm.utils

* address greptile review: fix yaml escaping, port validation, display name collisions, tests

- setup_wizard.py: add _yaml_escape() for safe YAML embedding of API keys
- setup_wizard.py: add _styled_input() with readline ANSI ignore markers
- setup_wizard.py: change DIVIDER to _divider() fn to avoid import-time color capture
- setup_wizard.py: validate port range 1-65535, initialize before loop
- setup_wizard.py: qualify azure display names (azure-gpt-4o) to avoid collision with openai
- setup_wizard.py: work on env_copy in _build_config to avoid mutating caller's dict
- setup_wizard.py: skip model_list entries for providers with no credentials
- setup_wizard.py: prompt for azure deployment name
- setup_wizard.py: wrap os.execlp in try/except with friendly fallback
- setup_wizard.py: wrap config write in try/except OSError
- setup_wizard.py: fix _validate_and_report to use two print lines (no \r overwrite)
- setup_wizard.py: add .gitignore tip next to key storage notice
- setup_wizard.py: fix run_setup_wizard() return type annotation to None
- scripts/install.sh: drop pipefail (not supported by dash on Ubuntu when invoked as sh)
- scripts/install.sh: use litellm[proxy] from PyPI (not hardcoded dev branch)
- scripts/install.sh: guard /dev/tty read with -r check for Docker/CI compat
- scripts/install.sh: remove --force-reinstall to avoid downgrading dependencies
- tests/test_litellm/test_setup_wizard.py: 13 unit tests for _build_config and _yaml_escape

* style: black format setup_wizard.py

* fix: address remaining greptile issues - Windows compat, YAML quoting, credential flow

- guard termios/tty imports with try/except ImportError for Windows compat
- quote master_key as YAML double-quoted scalar (same as env vars)
- remove unused port param from _build_config signature
- _validate_and_report now returns the final key so re-entered creds are stored
- add test for master_key YAML quoting

* fix: add --port to suggested command, guard /dev/tty exec in install.sh

* fix: quote api_base in YAML, skip azure if no deployment, only redraw on state change

* fix: address greptile review comments

- _yaml_escape: add control character escaping (\n, \r, \t)
- test: fix tautological assertion in test_build_config_azure_no_deployment_skipped
- test: add tests for control character escaping in _yaml_escape

* feat(ui): remove Chat UI page link and banner from sidebar and playground (#23908)

* feat(guardrails): MCPJWTSigner - built-in guardrail for zero trust MCP auth (#23897)

* Allow pre_mcp_call guardrail hooks to mutate outbound MCP headers

* Enhance MCPServerManager to support hook-modified arguments and extra headers. Update tests to validate argument mutation and header injection behavior, including warnings for OpenAPI-backed servers when headers are present.

* Refactor MCPServerManager to raise HTTPException for extra headers in OpenAPI-backed servers. Update tests to reflect this change, ensuring proper exception handling instead of logging warnings.

* Allow pre_mcp_call guardrail hooks to mutate outbound MCP headers

* Enhance MCPServerManager to support hook-modified arguments and extra headers. Update tests to validate argument mutation and header injection behavior, including warnings for OpenAPI-backed servers when headers are present.

* Refactor MCPServerManager to raise HTTPException for extra headers in OpenAPI-backed servers. Update tests to reflect this change, ensuring proper exception handling instead of logging warnings.

* feat(guardrails): add MCPJWTSigner built-in guardrail for zero trust MCP auth

Signs outbound MCP tool calls with a LiteLLM-issued RS256 JWT so MCP servers
can trust a single signing authority instead of every upstream IdP.

Enable in config.yaml:
  guardrails:
    - guardrail_name: mcp-jwt-signer
      litellm_params:
        guardrail: mcp_jwt_signer
        mode: pre_mcp_call
        default_on: true

JWT carries sub (user_id), act.sub (team_id, RFC 8693), tool-level scope, iss,
aud, iat/exp/nbf. RSA-2048 keypair auto-generated at startup unless
MCP_JWT_SIGNING_KEY env var is set.

Adds /.well-known/jwks.json endpoint and jwks_uri to /.well-known/openid-configuration
so MCP servers can verify LiteLLM-issued tokens via OIDC discovery.

* Update MCPServerManager to raise HTTPException with status code 400 for extra headers in OpenAPI-backed servers. Adjust tests to verify the correct status code and exception message.

* fix: address P1 issues in MCPJWTSigner

- OpenAPI servers: warn + skip header injection instead of 500
- JWKS Cache-Control: 5min for auto-generated keys, 1h for persistent
- sub claim: fallback to apikey:{token_hash} for anonymous callers
- ttl_seconds: validate > 0 at init time

* docs: add MCP zero trust auth guide with architecture diagram

* docs: add FastMCP JWT verification guide to zero trust doc

* fix: address remaining Greptile review issues (round 2)

- mcp_server_manager: warn when hook Authorization overwrites existing header
- __init__: remove _mcp_jwt_signer_instance from __all__ (private internal)
- discoverable_endpoints: copy dict instead of mutating in-place on OIDC augmentation
- test docstring: reflect warn-and-continue behavior for OpenAPI servers
- test: update scope assertions for least-privilege (no mcp:tools/list on tool-call JWTs)

* fix: address Greptile round 3 feedback

- initialize_guardrail: validate mode='pre_mcp_call' at init time — misconfigured
  mode silently bypasses JWT injection, which is a zero-trust bypass
- _build_claims: remove duplicate inline 'import re' (module-level import already present)
- _types.py: add TODO comment explaining jwt_claims is forward-compat plumbing
  for a follow-up PR that will forward upstream IdP claims into outbound MCP JWTs

* feat(mcp_jwt_signer): add verify+re-sign, claim ops, two-token model, configurable scopes

Addresses all missing pieces from the scoping doc review:

FR-5 (Verify + re-sign): MCPJWTSigner now accepts access_token_discovery_uri
and token_introspection_endpoint.  When set, the incoming Bearer token is
extracted from raw_headers (threaded through pre_call_tool_check), verified
against the IdP's JWKS (JWT) or introspected (opaque), and only re-signed if
valid.  Falls back to user_api_key_dict.jwt_claims for LiteLLM JWT-auth mode.

FR-12 (Configurable end-user identity mapping): end_user_claim_sources
ordered list drives sub resolution — sources: token:<claim>, litellm:user_id,
litellm:email, litellm:end_user_id, litellm:team_id.

FR-13 (Claim operations): add_claims (insert-if-absent), set_claims (always
override), remove_claims (delete) applied in that order.

FR-14 (Two-token model): channel_token_audience + channel_token_ttl issue a
second JWT injected as x-mcp-channel-token: Bearer <token>.

FR-15 (Incoming claim validation): required_claims raises HTTP 403 when any
listed claim is absent; optional_claims passes listed claims from verified
token into the outbound JWT.

FR-9 (Debug headers): debug_headers: true emits x-litellm-mcp-debug with kid,
sub, iss, exp, scope.

FR-10 (Configurable scopes): allowed_scopes replaces auto-generation.  Also
fixed: tool-call JWTs no longer grant mcp:tools/list (overpermission).

P1 fixes:
- proxy/utils.py: _convert_mcp_hook_response_to_kwargs merges rather than
  replaces extra_headers, preserving headers from prior guardrails.
- mcp_server_manager.py: warns when hook injects Authorization alongside a
  server-configured authentication_token (previously silent).
- mcp_server_manager.py: pre_call_tool_check now accepts raw_headers and
  extracts incoming_bearer_token so FR-5 verification has the raw token.
- proxy/utils.py: remove stray inline import inspect inside loop (pre-existing
  lint error, now cleaned up).

Tests: 43 passing (28 new tests covering all FR flags + P1 fixes).

* feat(mcp_jwt_signer): add verify+re-sign, claim ops, two-token model, configurable scopes (core)

Remaining files from the FR implementation:

mcp_jwt_signer.py — full rewrite with all new params:
  FR-5:  access_token_discovery_uri, token_introspection_endpoint,
         verify_issuer, verify_audience + _verify_incoming_jwt(),
         _introspect_opaque_token()
  FR-12: end_user_claim_sources ordered resolution chain
  FR-13: add_claims, set_claims, remove_claims
  FR-14: channel_token_audience, channel_token_ttl → x-mcp-channel-token
  FR-15: required_claims (raises 403), optional_claims (passthrough)
  FR-9:  debug_headers → x-litellm-mcp-debug
  FR-10: allowed_scopes; tool-call JWTs no longer over-grant tools/list

mcp_server_manager.py:
  - pre_call_tool_check gains raw_headers param to extract incoming_bearer_token
  - Silent Authorization override warning fixed: now fires when server has
    authentication_token AND hook injects Authorization

tests/test_mcp_jwt_signer.py:
  28 new tests covering all FR flags + P1 fixes (43 total, all passing)

* fix(mcp_jwt_signer): address pre-landing review issues

- Remove stale TODO comment on UserAPIKeyAuth.jwt_claims — the field is
  already populated and consumed by MCPJWTSigner in the same PR
- Fix _get_oidc_discovery to only cache the OIDC discovery doc when
  jwks_uri is present; a malformed/empty doc now retries on the next
  request instead of being permanently cached until proxy restart
- Add FR-5 test coverage for _fetch_jwks (cache hit/miss),
  _get_oidc_discovery (cache/no-cache on bad doc), _verify_incoming_jwt
  (valid token, expired token), _introspect_opaque_token (active,
  inactive, no endpoint), and the end-to-end 401 hook path — 53 tests
  total, all passing

* docs(mcp_zero_trust): rewrite as use-case guide covering all new JWT signer features

Add scenario-driven sections for each new config area:
- Verify+re-sign with Okta/Azure AD (access_token_discovery_uri,
  end_user_claim_sources, token_introspection_endpoint)
- Enforcing caller attributes with required_claims / optional_claims
- Adding metadata via add_claims / set_claims / remove_claims
- Two-token model for AWS Bedrock AgentCore Gateway
  (channel_token_audience / channel_token_ttl)
- Controlling scopes with allowed_scopes
- Debugging JWT rejections with debug_headers

Update JWT claims table to reflect configurable sub (end_user_claim_sources)

* fix(mcp_jwt_signer): wire all config.yaml params through initialize_guardrail

The factory was only passing issuer/audience/ttl_seconds to MCPJWTSigner.
All FR-5/9/10/12/13/14/15 params (access_token_discovery_uri,
end_user_claim_sources, add/set/remove_claims, channel_token_audience,
required/optional_claims, debug_headers, allowed_scopes, etc.) were
silently dropped, making every advertised advanced feature non-functional
when loaded from config.yaml.

Add regression test that asserts every param is wired through correctly.

* docs(mcp_zero_trust): add hero image

* docs(mcp_zero_trust): apply Linear-style edits

- Lead with the problem (unsigned direct calls bypass access controls)
- Shorter statement section headers instead of question-form headers
- Move diagram/OIDC discovery block after the reader is bought in
- Add 'read further only if you need to' callout after basic setup
- Two-token section now opens from the user problem not product jargon
- Add concrete 403 error response example in required_claims section
- Debug section opens from the symptom (MCP server returning 401)
- Lowercase claims reference header for consistency

* fix(mcp_jwt_signer): fix algorithm confusion attack + add OIDC discovery 24h TTL

- Remove alg from unverified JWT header; use signing_jwk.algorithm_name from JWKS key instead.
  Reading alg from attacker-controlled headers enables alg:none / HS256 confusion attacks.
- Add _oidc_discovery_fetched_at timestamp and _OIDC_DISCOVERY_TTL = 86400 (24h).
  Without a TTL the cached discovery doc never refreshes, so IdP key rotation is invisible.

---------

Co-authored-by: Noah Nistler <60981020+noahnistler@users.noreply.github.com>

* fix(ci): stabilize CI - formatting, type errors, test polling, security CVEs, router bug, batch resolution

Fix 1: Run Black formatter on 35 files
Fix 2: Fix MyPy type errors:
  - setup_wizard.py: add type annotation for 'selected' set variable
  - user_api_key_auth.py: remove redundant type annotation on jwt_claims reassignment
Fix 3: Fix spend accuracy test burst 2 polling to wait for expected total
  spend instead of just 'any increase' from burst 2
Fix 4: Bump Next.js 16.1.6 -> 16.1.7 to fix CVE-2026-27978, CVE-2026-27979,
  CVE-2026-27980, CVE-2026-29057
Fix 5: Fix router _pre_call_checks model variable being overwritten inside
  loop, causing wrong model lookups on subsequent deployments. Use local
  _deployment_model variable instead.
Fix 6: Add missing resolve_output_file_ids_to_unified call in batch retrieve
  non-terminal-to-terminal path (matching the terminal path behavior)

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* chore: regenerate poetry.lock to sync with pyproject.toml

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix: format merged files from main and regenerate poetry.lock

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix(mypy): annotate jwt_claims as Optional[dict] to fix type incompatibility

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix(ci): update router region test to use gpt-4.1-mini (fix flaky model lookup)

Replace deprecated gpt-3.5-turbo-1106 with gpt-4.1-mini + mock_response in
test_router_region_pre_call_check, following the same pattern used in commit
717d37cc5b for test_router_context_window_check_pre_call_check_out_group.

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* ci: retry flaky logging_testing (async event loop race condition)

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix(ci): aggregate all mock calls in langfuse e2e test to fix race condition

The _verify_langfuse_call helper only inspected the last mock call
(mock_post.call_args), but the Langfuse SDK may split trace-create and
generation-create events across separate HTTP flush cycles. This caused
an IndexError when the last call's batch contained only one event type.

Fix: iterate over mock_post.call_args_list to collect batch items from
ALL calls. Also add a safety assertion after filtering by trace_id and
mark all langfuse e2e tests with @pytest.mark.flaky(retries=3) as an
extra safety net for any residual timing issues.

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix(ci): black formatting + update OpenAPI compliance tests for spec changes

- Apply Black 26.x formatting to litellm_logging.py (parenthesized style)
- Update test_input_types_match_spec to follow $ref to InteractionsInput schema
  (Google updated their OpenAPI spec to use $ref instead of inline oneOf)
- Update test_content_schema_uses_discriminator to handle discriminator without
  explicit mapping (Google removed the mapping key from Content discriminator)

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* revert: undo incorrect Black 26.x formatting on litellm_logging.py

The file was correctly formatted for Black 23.12.1 (the version pinned
in pyproject.toml). The previous commit applied Black 26.x formatting
which was incompatible with the CI's Black version.

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix(ci): deduplicate and sort langfuse batch events after aggregation

The Langfuse SDK may send the same event (e.g., trace-create) in
multiple flush cycles, causing duplicates when we aggregate from all
mock calls. After filtering by trace_id, deduplicate by keeping only
the first event of each type, then sort to ensure trace-create is at
index 0 and generation-create at index 1.

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Noah Nistler <60981020+noahnistler@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
2026-03-18 15:09:01 -07:00
yuneng-jiang
2c0c08722a
Merge pull request #24035 from BerriAI/litellm_fix_guardrail_mode_type_crash
[Fix] UI - Logs: Guardrail Mode Type Crash on Non-String Values
2026-03-18 15:05:52 -07:00
yuneng-jiang
51f78b7d72 address greptile review feedback (greploop iteration 4)
- Add guard assertion before non-null click on custom code switch
- Use await act(async ...) for timer advancement to avoid act warnings
- Pin locale in date range assertion for CI determinism

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 13:09:17 -07:00
yuneng-jiang
eb7efa36da Add missing permission options to PERMISSION_OPTIONS list
Adds /key/info, /key/list, /key/aliases, and /team/daily/activity
to the hardcoded PERMISSION_OPTIONS in TeamSSOSettings.tsx.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 13:05:53 -07:00
yuneng-jiang
b75266d254 merge origin/main, resolve test file conflicts
Resolved conflicts in ScoreChart.test.tsx and HelpLink.test.tsx by
preferring origin/main's renderWithProviders pattern and merging
unique tests from both branches.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 12:57:31 -07:00
yuneng-jiang
bbc120095e address greptile review feedback (greploop iteration 3)
- Remove Ant Design CSS class selector coupling in ExportFormatSelector test
- Lift mock fns out of TestTable component body to enable callback assertions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 12:48:01 -07:00
yuneng-jiang
56ed8379e2 address greptile review feedback (greploop iteration 2)
- Add explicit vi import to ScoreChart.test.tsx
- Use custom matcher for I/O modes to avoid cross-element text issues
- Use version-agnostic regex for Save button assertion
- Add comments noting placeholder data in GuardrailConfig tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 12:42:19 -07:00
yuneng-jiang
7714d1be0b address greptile review feedback (greploop iteration 3)
Add typeof string guards to all array element returns in resolveMode
to prevent non-string values from sneaking through via any-widening.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 12:40:56 -07:00
yuneng-jiang
6902355f5b address greptile review feedback (greploop iteration 2)
Replace unsafe `as string[]` cast in modeMatches with runtime type
check via `.some()`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 12:34:03 -07:00
yuneng-jiang
40721ab18f address greptile review feedback (greploop iteration 1)
- Move vi.useRealTimers() to afterEach for proper cleanup
- Use label-based DOM queries instead of fragile positional indexes
- Remove leftover debug console.log from AgentHubTableColumns.tsx

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 12:31:25 -07:00
yuneng-jiang
1c8b5f77c9 address greptile review feedback (greploop iteration 1)
Add modeMatches() helper so array guardrail_mode values (e.g.
["pre_call", "post_call"]) place the entry in all matching timeline
buckets, not just the first. Updated test to verify both buckets.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 12:29:57 -07:00
yuneng-jiang
20c1d984a6 [Test] UI: Add unit tests for 10 previously untested components
Add Vitest + RTL tests covering DebugWarningBanner, HelpLink, ExportFormatSelector, ExportSummary, ExportTypeSelector, UsageExportHeader, MetricCard, ScoreChart, GuardrailConfig, and AgentHubTableColumns. 73 tests total.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 12:18:42 -07:00
yuneng-jiang
b13a7c6790 Fix guardrail_mode.replace crash when backend returns non-string value
The backend type for guardrail_mode is Optional[Union[str, List[str], Dict]]
but the UI typed it as just string, causing a crash when .replace() was
called on null/object/array values.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 12:17:19 -07:00
yuneng-jiang
441f768abd Merge remote-tracking branch 'origin' into litellm_yj_march_17_2026 2026-03-18 12:07:50 -07:00
yuneng-jiang
bbabdaab38
Merge pull request #23973 from BerriAI/litellm_/fervent-hypatia
[Refactor] UI - Playground: Extract FilePreviewCard from ChatUI
2026-03-18 00:44:17 -07:00
yuneng-jiang
cd549bf4f5 [Refactor] UI - Playground: Extract FilePreviewCard from ChatUI
Extract duplicate file preview JSX blocks (responses and chat image
previews) into a reusable FilePreviewCard component, reducing ~50
lines of duplicated markup in ChatUI.tsx.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 00:40:00 -07:00
yuneng-jiang
62835ff03d adding package-lock 2026-03-17 17:44:01 -07:00
yuneng-jiang
3e2845181c bumping next version 2026-03-17 17:38:09 -07:00
Rohan
bed44f5fe5
Add Akto Guardrails to LiteLLM (#23250)
* akto guardrails support in litellm

* docs(guardrails): add akto to supported values in types/guardrails.py

* frontend changes + fixes

* feat(akto): update Akto guardrail integration with new configuration options and modes

* docs(akto): enhance Akto documentation and configuration descriptions for clarity

* feat(tests): add proxy server request headers to sample request data

* refactor(akto): remove optional account and VXLAN IDs; update documentation and tests

* feat(akto): add event_type parameter for enhanced observability in guardrail logging

* refactor(akto): update environment variable references

* refactor the python codes

* refactor and fix linting

* refactor(akto): remove unused event hook and clean up imports

* refactor(akto): enhance AktoGuardrail with async support and improved logging

* fix: Register DynamoAI guardrail initializer and enum entry (#23752)

* fix: Register DynamoAI guardrail initializer and enum entry

Fix the "Unsupported guardrail: dynamoai" error by:
1. Adding DYNAMOAI to SupportedGuardrailIntegrations enum
2. Implementing initialize_guardrail() and registries in dynamoai/__init__.py

The DynamoAI guardrail was added in PR #15920 but never properly registered
in the initialization system. The __init__.py was missing the
guardrail_initializer_registry and guardrail_class_registry dictionaries
that the dynamic discovery mechanism looks for at module load time.

Fixes #22773

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

* Update litellm/proxy/guardrails/guardrail_hooks/dynamoai/__init__.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update litellm/proxy/guardrails/guardrail_hooks/dynamoai/__init__.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* test: Add tests for DynamoAI guardrail registration

Verifies enum entry, initializer registry, class registry,
instance creation, and global registry discovery.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* docs: add v1.82.3 release notes and update provider_endpoints_support.json (#23816)

* Revert "docs: add v1.82.3 release notes and update provider_endpoints_support…" (#23817)

This reverts commit 966124966f.

* Refactor Akto guardrail configuration and tests; update UI description and tags

* add account and vxlan ID parameters to Akto guardrail initialization; update Akto logo format

* enhance Akto guardrail documentation and improve error handling for non-JSON responses

* address greptile issues

* fix: update payload handling to use 'data' instead of 'json' in AktoGuardrail and adjust tests accordingly

---------

Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Joe Reyna <joseph.reyna@gmail.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2026-03-17 14:38:04 -07:00
yuneng-jiang
e0e1ac617a [Test] UI: Add unit tests for 10 previously untested components
Add Vitest + RTL tests for HelpLink, DebugWarningBanner, ExportFormatSelector,
ExportTypeSelector, ExportSummary, MetricCard, PolicySelect,
ComplexityRouterConfig, RateLimitTypeFormItem, and AgentCardGrid (67 tests total).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 13:30:06 -07:00
ryan-crabbe
ef9cc33ee3
Merge pull request #23822 from BerriAI/litellm_ryan_march_16
Litellm ryan's daily branch march 16
2026-03-17 10:03:01 -07:00
ryan-crabbe
cde28aa018
Merge pull request #23819 from BerriAI/litellm_fix-csv-export
fix(ui): CSV export empty on Global Usage page
2026-03-16 22:46:36 -07:00
Ryan Crabbe
c098eca509 fix(ui): CSV export empty on Global Usage page
Aggregated endpoint returns empty breakdown.entities; fall back to
grouping breakdown.api_keys by team_id.
2026-03-16 22:36:23 -07:00
yuneng-jiang
471e0f147e [Fix] Remove "API" from custom key description text
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 22:35:44 -07:00
yuneng-jiang
0b0fe7e263 [Fix] Rename toggle label to "Disable custom Virtual key values"
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 22:26:53 -07:00
yuneng-jiang
c687e631b4 [Feature] Add disable_custom_api_keys toggle to UI Settings page
Adds a toggle switch to the admin UI Settings page so administrators can
enable/disable custom API key values without making direct API calls.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 22:26:11 -07:00
yuneng-jiang
53d96c8353 [Feature] Disable custom API key values via UI setting
Add disable_custom_api_keys UI setting that prevents users from specifying
custom key values during key generation and regeneration. When enabled, all
keys must be auto-generated, eliminating the risk of key hash collisions
in multi-tenant environments.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 21:35:21 -07:00
yuneng-jiang
a771fe55e4 [Fix] Update log filter test to match empty-result behavior
The test expected fallback to all logs when backend filters return empty,
but the source was intentionally changed to show empty results instead of
stale data. Updated test to match.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 18:19:02 -07:00
yuneng-jiang
57bba3b863 [Fix] UI - Logs: Fix empty filter results showing stale data
Remove `.length > 0` check so that when a backend filter returns an
empty result set the table correctly shows no data instead of falling
back to the previous unfiltered logs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 16:28:11 -07:00
Ryan Crabbe
5befc025d8 chore(ui): use antd danger prop instead of tailwind for Remove button 2026-03-16 15:36:28 -07:00
Ryan Crabbe
016667b2fa chore(ui): migrate DefaultUserSettings buttons from Tremor to antd 2026-03-16 15:04:22 -07:00
yuneng-jiang
eba8df5235
Merge pull request #23773 from BerriAI/litellm_/reverent-panini
[Test] UI Dashboard - Add unit tests for 5 untested files
2026-03-16 14:42:16 -07:00
yuneng-jiang
5d33cc66a0 Add unit tests for 5 previously untested UI dashboard files
Tests added for: UiLoadingSpinner, HashicorpVaultEmptyPlaceholder,
PageVisibilitySettings, errorUtils, and mcpToolCrudClassification.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 12:53:20 -07:00
yuneng-jiang
8f56ddb9c6 Merge remote main into litellm_ci_optimize
Resolved conflict in test_claude_agent_sdk.py by keeping main's additions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 00:50:22 -07:00
yuneng-jiang
eb7222c7cb [Infra] Reduce CI no_output_timeout and tighten Playwright timeouts
- Change no_output_timeout from 120m/60m to 15m across all test jobs
- Fix multi-line YAML formatting for no_output_timeout entries
- Reduce Playwright per-test timeout from 4min to 3min
- Add 15s actionTimeout and 30s navigationTimeout to Playwright config

Hanging tests now fail in 15min instead of 2hrs, cutting wasted CI time.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 19:38:10 -07:00
yuneng-jiang
82fc819abf Merge remote-tracking branch 'origin' into litellm_internal_dev_03_14_2026 2026-03-14 18:35:03 -07:00
yuneng-jiang
a2708f332f
Merge pull request #23658 from BerriAI/litellm_internal_dev_03_13_2026
[Infra] Merge internal dev 3/13 with main
2026-03-14 14:22:27 -07:00
yuneng-jiang
2c840f1e8c
Merge pull request #23622 from BerriAI/litellm_usage_page_auto_pagination
[Feature] UI - Usage: Auto-paginate daily spend data
2026-03-14 11:44:05 -07:00
yuneng-jiang
d26faeb844 [Fix] UI - Usage: Reduce batch size to 3, add loading spinner to fetch banner
- Reduce RENDER_BATCH_SIZE from 5 to 3 for more frequent chart updates
- Add LoadingOutlined spinner at the start of all fetching Alert banners

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 11:28:15 -07:00
yuneng-jiang
6b783cc0c5
Merge pull request #23614 from BerriAI/litellm_team_default_permissions
[Feature] UI - Default Team Settings: Modernize page and fix defaults application
2026-03-14 11:12:07 -07:00
yuneng-jiang
f72931a463 [Feature] UI - Usage: Prominent fetch banner, batched pagination renders
Replace subtle loading text with antd Alert banners that clearly communicate
pagination status, and batch state flushes to reduce chart re-renders.

- Replace inline loading text with warning Alert banners showing progress,
  "open a new tab" link with ExportOutlined icon, and primary Stop button
- Batch setData calls every 5 pages instead of per-page to cut re-renders ~80%
- Reduce fetch delay from 500ms to 300ms for faster data loading
- Add "Charts will update periodically" messaging to set expectations
- Fix pre-existing TS error: Button icon prop was using render function instead of ReactNode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 11:09:10 -07:00
yuneng-jiang
e8e46af8b0
Merge pull request #23638 from BerriAI/litellm_ui_user_info_teams
[Feature] UI - Internal Users: Add/Remove Team Membership
2026-03-14 10:12:57 -07:00
yuneng-jiang
db37f31099 [Fix] Address review feedback on paginated daily activity hook
1. Replace ...args spread in useEffect deps with JSON.stringify(args) key
   to prevent infinite re-renders when callers pass unstable array references.
2. Add missing agentCancelled partial-data message in EntityUsage so the
   outer condition no longer renders an empty div.
3. Store setTimeout ID in a ref and clearTimeout on cleanup/cancel to avoid
   orphaned timers under rapid re-renders.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 09:55:57 -07:00
yuneng-jiang
0cd4a68157 [Fix] Add missing networking mocks to CreateKeyPage test
The test's partial vi.mock of @/components/networking was missing the daily
activity call exports now imported by EntityUsage via ENTITY_FETCH_FNS.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 09:47:56 -07:00
yuneng-jiang
36c076d56c
Merge pull request #23550 from Sameerlite/litellm_azure-model-router-cost-breakdown
feat(azure): Azure Model Router cost breakdown in UI + additional_costs from hidden_params
2026-03-14 09:39:04 -07:00
yuneng-jiang
83e6096dae [Feature] UI - Internal Users: Add/remove team membership from user info page
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 09:37:44 -07:00
yuneng-jiang
3ae9e070d9 [Feature] UI - Usage: Auto-paginate daily spend data with progressive rendering
Previously, EntityUsage only fetched page 1 of paginated daily spend endpoints,
showing incomplete data. UsagePageView fetched all pages but blocked the UI until
completion. This adds a reusable usePaginatedDailyActivity hook that fetches pages
sequentially with 500ms delays, updates charts progressively, and supports
cancellation on unmount or user action.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 00:30:03 -07:00