Fixes issue where multipart file uploads through passthrough endpoints failed with RequestValidationError. The proxy was consuming the request body stream and FastAPI was trying to parse multipart bodies as JSON dicts.
Changes:
- Try JSON parsing first for multipart content-type (handles misconfigured clients)
- Skip multipart parsing if JSON succeeds to avoid stream consumption
- Remove custom_body parameter from endpoint_func to prevent FastAPI auto-parsing
- Check for parsed body before using multipart handler
- Add regression test for multipart boundary preservation
Handles both actual multipart uploads and JSON bodies with incorrect multipart content-type headers.
Made-with: Cursor
The GET /v1/mcp/server endpoint allowed any authenticated user to pass
an arbitrary team_id and enumerate another team's MCP server config.
Restricted virtual keys could also use the team_id param to bypass
their access limitations. Add team membership check for non-admins
and block restricted keys from using the team_id filter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When creating a key, the MCP server list now filters by the selected team's
allowed servers. Also fixes UnboundLocalError on `is_restricted_virtual_key`
when `team_id` query param was provided to GET /v1/mcp/server.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(mcp): add BYOM (Bring Your Own MCPs) submission + admin review workflow
Non-admins can now submit MCP servers for review via POST /v1/mcp/server/register.
Admins get a Submissions tab in the UI to approve or reject pending servers.
Approved servers enter the active runtime; rejected ones stay out with notes.
- DB: add approval_status, submitted_by, submitted_at, reviewed_at, review_notes
to LiteLLM_MCPServerTable with migration
- Backend: new endpoints register, submissions, approve, reject
- reload_servers_from_database now only loads approval_status=active servers
- UI: Submissions tab with stat cards, card list, confirm dialogs; non-admin
"Submit MCP Server" button wired to /register endpoint
- Fix get_mcp_submissions to filter by submitted_at IS NOT NULL (not submitted_by,
which can be null for team-scoped keys without an associated user)
* feat(mcp): rename nav item to Team MCPs + add New badge
* fix(mcp): revert nav label, rename Submissions tab to Team MCPs + New badge
* feat(mcp): add MCP Standards — required fields config + CI-style checks on submissions
Adds a "Standards" tab (admin-only) to MCP Servers where admins define which
server fields are required for a submission to pass. Each submission card in
Team MCPs then shows a green ✓ or red ✗ for each required field, with a
summary "N/M checks" badge in the header — like GitHub CI status rows.
Also adds a `source_url` field (GitHub / Source URL) to the MCP server schema
so non-admins can link to the source repo when submitting a server.
- schema.prisma: add `source_url String?` to LiteLLM_MCPServerTable
- migration: 20260309000001_add_mcp_source_url
- _types.py: source_url on NewMCPServerRequest, UpdateMCPServerRequest, LiteLLM_MCPServerTable
- types.tsx: source_url on MCPServer interface
- create_mcp_server.tsx: GitHub/Source URL form field
- MCPStandardsSettings.tsx: new — toggle which fields are required (stored in general settings as mcp_required_fields)
- mcp_servers.tsx: Standards tab (admin-only)
- MCPSubmissionsTab.tsx: load required fields + CI-style check pills on each card
* refactor(mcp): move submission rules into Team MCPs tab, grouped free-form UI
Folds the Standards tab into Team MCPs. Submission Rules panel now lives at the
top of the Team MCPs tab — collapsible, shows active rules as chips when closed,
expands to a grouped checkbox editor (Documentation / Source / Connection /
Security). Removes the separate Standards tab from the nav.
MCPStandardsSettings.tsx is now constants-only (FIELD_GROUPS, MCP_REQUIRED_FIELD_DEFS,
SETTINGS_KEY) — the UI lives in MCPSubmissionsTab.
* feat(mcp): add mcp_required_fields to ConfigGeneralSettings + config/list endpoint
Registers mcp_required_fields as a proper general_settings field so the UI
can read/write it via /config/list and /config/field/update without the
"Invalid field" error. Also fixes a pre-existing pyright None-check issue
in _sync_ui_settings_to_general_settings.
* ui(mcp): GitHub-style PR checks panel on submission cards
* ui: rename Team MCPs -> Submitted Tools, Team Guardrails -> Submitted Guardrails
* address greptile review feedback (greploop iteration 1)
* fix: inline import, add approval workflow tests, rename Submitted MCPs
* fix(mcp): allow re-approval of rejected MCP server submissions
* fix(mcp): evict rejected servers from runtime; enforce mcp_required_fields on /register
* fix(mcp): sort submissions newest-first; force active status on admin-created servers
* fix(mcp): add missing mock in test, show Approve for rejected, clear submission metadata, drop spurious Content-Type
* fix(mcp/ui): show Reject for active servers; show submit form to non-admins with team-key note
* fix(mcp): conditional reload on reject; view-only admin for submissions; block admin from /register
* fix(mcp): match auth_type required-field validation to UI compliance check (reject 'none')
* fix(mcp): block view-only admin from /register; log settings failure; warn on active server reject
* fix(mcp): allow view-only admin to use /register; add _validate_mcp_required_fields tests
* fix(mcp): validate field names in mcp_required_fields; surface backend error in submit UI
* fix(mcp): fix falsy field check; add field-name validation; add take limit; document server-managed fields; close dialog on error
* fix(mcp): add AWS SigV4 auth for Bedrock AgentCore MCP servers
Add aws_sigv4 auth type to MCP client via httpx.Auth subclass that
signs each request with SigV4 using botocore. Enables mcp_servers
config to connect to AgentCore-hosted MCP servers.
* docs(mcp): add AWS SigV4 auth documentation for Bedrock AgentCore
Add dedicated docs page for configuring MCP servers with AWS SigV4
authentication, update MCP overview with aws_sigv4 auth type and
config example, and link from Bedrock AgentCore provider docs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(mcp): address Greptile review — requires_request_body, full header signing, health check
- Add requires_request_body = True to MCPSigV4Auth so httpx buffers the
request body before calling auth_flow (prevents empty body hash for
streaming requests)
- Pass all request headers to AWSRequest for canonical SigV4 signing
instead of only Content-Type
- Exclude aws_sigv4 from health check skip logic since it has its own
credential fields (not authentication_token)
- Fix docs: mark aws_access_key_id/aws_secret_access_key as optional
(falls back to boto3 credential chain)
- Add test for requires_request_body flag
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
- Fix case-insensitive tool name matching in _tool_name_matches() so that
OpenAPI operationIds (camelCase) match lowercase registered tool names
when filtering by allowed_tools
- Fix get_base_url() to resolve relative server URLs (e.g. /api/v3) by
deriving full base URL from spec_path when OpenAPI spec has relative URLs
- Add tests for case-insensitive matching and filter_tools_by_allowed_tools
Made-with: Cursor
`all_models = user_api_key_dict.models` was creating an alias, so
`_get_models_from_access_groups` (which uses `.pop()`/`.extend()`) would
mutate the cached object in-place. Now both `.models` and `.team_models`
assignments create copies via `list()`.
Added test to verify the input is not mutated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds dedup to get_key_models and get_team_models to prevent duplicate
entries when access group member models overlap with proxy_model_list.
Removes dead assignment of all_models in get_team_models.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a team has "all-proxy-models", the model list expansion now includes
model access group names so they appear in the UI key creation form.
Also fixes get_key_models not forwarding include_model_access_groups to
_get_models_from_access_groups, and removes unused _unfurl_all_proxy_models.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The tag daily activity endpoint used compute_tag_metadata_totals which
deduplicates by request_id, but LiteLLM_DailyTagSpend stores aggregated
daily records where request_id is either NULL or stale. This caused all
metadata totals (total_spend, total_requests, etc.) to be 0 in the UI
cost panel. Now uses the same standard totals as every other entity type.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Audit logs (CRUD events on keys, teams, users, models) were only stored in
the Prisma DB. This adds a pluggable callback system so audit logs can be
forwarded to external services like S3 for ingestion into security monitoring
tools.
New config key `audit_log_callbacks` under `litellm_settings` reuses the
existing callback infrastructure. Any CustomLogger subclass can opt in by
overriding `async_log_audit_log_event()`. S3Logger (s3_v2) is implemented
as the first handler, storing audit logs under `audit_logs/{date}/` prefix.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(proxy): add Prisma DB pool and engine health metrics to Prometheus
Add a PrismaMetricsCollector that periodically queries pg_stat_activity
and the Prisma engine process to expose connection pool and engine health
as Prometheus gauges/counters. Auto-enabled when prometheus_system is in
service_callback.
New metrics:
- litellm_db_pool_active_connections (Gauge)
- litellm_db_pool_idle_connections (Gauge)
- litellm_db_pool_total_connections (Gauge)
- litellm_db_pool_waiting_connections (Gauge)
- litellm_db_engine_up (Gauge)
- litellm_db_engine_restarts_total (Counter)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address Greptile review feedback
- Only increment engine_restarts counter on heavy reconnects (engine
actually dead), not lightweight network-blip reconnects
- Fix potential KeyError in _get_or_create_gauge/counter fallback path
when REGISTRY._names_to_collectors is absent
- Rename litellm_db_pool_waiting_connections to
litellm_db_pool_lock_waiting_connections to clarify it measures lock
contention, not pool slot queuing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: warn when prometheus_system enabled but watchdog disabled
Log a warning when users have prometheus_system in service_callback
but PRISMA_HEALTH_WATCHDOG_ENABLED=false, since DB pool and engine
metrics won't be collected in that configuration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* ci: retrigger CI checks
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: use labeled gauge for DB pool connection metrics
Replace 3 separate pool gauges (active, idle, total) with a single
`litellm_db_pool_connections` gauge using a `state` label. This is more
Prometheus-idiomatic and exposes all pg_stat_activity states (active,
idle, idle in transaction, etc.) without ambiguity about what "total"
includes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address Greptile review — stale labels and fallback re-registration
- Zero out known pg_stat_activity states that are absent from the current
query result, preventing stale gauge values from persisting.
- Simplify _get_or_create_gauge/counter by removing the fallback loop
that could re-register an already-registered metric (ValueError).
- Add test for stale label clearing across collection cycles.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: include "unknown" in _PG_STATES for stale label clearing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: collect immediately on start and consolidate into single query
- Move sleep to end of loop so metrics appear on /metrics immediately
after startup instead of after a 30s delay.
- Combine pool state and lock waiting queries into a single SQL query
using conditional aggregation, halving per-cycle DB overhead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent tight spin loop on collection error
Move asyncio.sleep outside the try/except so it always executes even
when _collect_engine_health() or _collect_pool_metrics() raises.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add multiprocess_mode to _get_or_create_gauge initialization
- Include `multiprocess_mode` parameter to properly support multiprocessing in Gauge creation.
- Ensure consistent behavior for labeled and unlabeled Gauges.
* fix: handle invalid env var and document watchdog prerequisite
- Add try/except ValueError for PRISMA_METRICS_COLLECTION_INTERVAL_SECONDS
to prevent proxy startup crash on non-numeric values (e.g. "30s")
- Document that DB metrics require both prometheus_system callback and
PRISMA_HEALTH_WATCHDOG_ENABLED=true
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use defensive null coalescing for query_raw row values
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add invalid env var fallback test and fix mock signature
- Add test for non-numeric PRISMA_METRICS_COLLECTION_INTERVAL_SECONDS
- Add **kwargs to mock _patched_get_or_create_gauge for forward compat
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
When both enable_jwt_auth and enable_oauth2_auth are True, the proxy now
routes tokens based on their format:
- JWT tokens (3 dot-separated parts) -> JWT auth handler
- Opaque tokens -> OAuth2 auth handler
This enables using JWT for human users and OAuth2 for M2M (machine) clients
on the same LiteLLM instance. Previously, enabling OAuth2 would intercept
all tokens on LLM API routes before JWT auth could run.
When only one auth method is enabled, behavior is unchanged (backward compatible).
When calling get_users() directly (not via FastAPI), Query() defaults
are not resolved. Pass organization_ids=None explicitly to avoid
'Query' object has no attribute 'split' error.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- test_get_users_*: pass proxy admin user_api_key_dict since get_users
now calls _authorize_user_list_request which checks user_role
- test_validate_team_member_add_permissions_non_admin: set
organization_id on mock team since _is_user_org_admin_for_team
accesses it
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Thread org objects {organization_id, organization_alias} instead of bare IDs from
users/page.tsx → view_users.tsx → CreateUserButton so the selector can show aliases
- Replace single-select org dropdown with multi-select; always shown when organizationIds
is non-null; disabled/pre-selected for single-org admins; displays "Alias (id)"
- handleCreate: maps organization_ids → organizations before POST, removes redundant
organizationMemberAddCall (backend _add_user_to_organizations handles it)
- _user_is_org_admin: also checks organizations list field in addition to singular
organization_id so /user/new succeeds for org admins
- Add 5 backend unit tests for _user_is_org_admin and 2 frontend tests for new form behavior
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
