- Azure Model Router transform_response: let parent extract actual model from raw response
- common_request_processing: skip model override for Azure Model Router requests
- proxy_server: skip streaming chunk model restamp for Azure Model Router
- Add _is_azure_model_router_request helper
- Add tests for non-streaming and streaming
Made-with: Cursor
* fix(mcp): OAuth2 chat connect - tools fetch, auth flow, and status fixes
- schema.prisma: add missing MCP table fields (approval_status, submitted_by, submitted_at, reviewed_at, review_notes) to prevent destructive migrations
- rest_endpoints.py: inject user OAuth token via extra_headers for OAuth2 servers so tools list is populated; add server name->UUID resolution so MCPConnectPicker name lookups work
- mcp_registry.json: fix Atlassian defaults (transport: http, url: .../v1/mcp)
- ChatPage.tsx: read mcpOauthReturn param to init sidebarView="apps" on OAuth return, clean up param after mount
- MCPAppsPanel.tsx: auto-add OAuth2 servers to selectedServers when credential detected; onConnect also enables server for chat; disconnect removes from selectedServers
- mcp_servers.tsx: sort servers by created_at DESC
- useUserMcpOAuthFlow.tsx: append mcpOauthReturn=apps to return URL so Apps panel is mounted on return
* address greptile review feedback (greploop iteration 1)
* fix(mcp): inject stored OAuth2 token when fetching tools via /responses API
When a user has connected an OAuth2 MCP server (e.g. Atlassian) and then
uses the /responses endpoint with that server, tool listing was failing
because the stored per-user OAuth token was never injected.
Two fixes:
1. server.py: add _get_user_oauth_extra_headers_from_db() helper; call it
in _get_tools_from_mcp_servers when oauth2_headers is None for an OAuth2
server, falling back to the user's stored token in LiteLLM_MCPUserCredentials
2. litellm_proxy_mcp_handler.py: also intercept MCP tools whose server_url
matches */mcp/<server_name> (e.g. http://localhost:4000/mcp/atlassian_test)
by rewriting them to litellm_proxy/mcp/<server_name> so they go through
the internal handler (and get the OAuth token injected) instead of being
forwarded to OpenAI raw where localhost is unreachable
* address greptile review feedback (greploop iteration 2)
* test(mcp): add unit test for OAuth2 token injection in _get_tools_from_mcp_servers
Verifies that when _get_tools_from_mcp_servers is called for an OAuth2 MCP
server without oauth2_headers in the request, the implementation:
- calls _prefetch_oauth_creds_for_user once (not per-server) to avoid N+1 queries
- passes the stored token as extra_headers={"Authorization": "Bearer ..."} to
_get_tools_from_server so the upstream OAuth2 MCP server authenticates correctly
* address greptile review feedback (greploop iteration 3)
* address greptile review feedback (greploop iteration 4)
* address greptile review feedback (greploop iteration 5)
* redesign credentials table to use Tremor table layout matching Keys page
* fix: /server/oauth authorize 422 - make client_id optional, fall back to real DB server
* fix: mcp_token client_id optional, resolve from server record
* fix: look up real server by UUID (get_mcp_server_by_id) before falling back to name
* Update litellm/responses/mcp/litellm_proxy_mcp_handler.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: address greptile feedback - client_id guards, dict spread, helper refactor, tests
- mcp_management_endpoints: raise 400 when resolved_client_id is empty in
mcp_authorize and mcp_token instead of forwarding "" to upstream
- litellm_proxy_mcp_handler: use {**tool, "server_url": ...} spread instead
of dict(tool) + mutation for shallow copy safety
- rest_endpoints: extract _oauth2_server_ids set comprehension to a named
_get_oauth2_server_ids() helper for clarity; add Set to typing imports
- test_rest_endpoints: add tests for name→UUID resolution path,
access-denied when resolved UUID not in allowed list, and OAuth2 user
token injection for single-server requests; fix fake_get_tools signature
to accept extra_headers kwarg
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Snowflake's Cortex LLM API (like Anthropic) requires tool_choice as an
object with a "type" field, not as a bare string. Passing tool_choice="auto"
(or "required"/"none") results in error 390142 "invalid payload".
This fix transforms OpenAI string tool_choice values to the Snowflake
object format:
- "auto" -> {"type": "auto"}
- "required" -> {"type": "any"} (Snowflake/Anthropic convention)
- "none" -> {"type": "none"}
The dict-to-dict transformation for specific function tool choices
({"type": "function", "function": {"name": "..."}} -> {"type": "tool",
"name": [...]}) remains unchanged.
Fixes#23284
Co-authored-by: gambletan <tan@echooo.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
- Replace gemini-pro with gemini-3-pro-preview in test_cost_discount_vertex_ai
(gemini-pro removed from cost map)
- Replace github/claude-3-5-sonnet-latest with github/claude-3-7-sonnet-20250219
in test_supports_function_calling_github_anthropic_alias (model removed)
- Add supports_multimodal, uses_embed_content, input/output_cost_per_token_above_256k_tokens
to JSON schema in test_utils.py (new properties added to model cost map)
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
- Add black_forest_labs and charity_engine to provider_endpoints_support.json
(fixes check_code_and_doc_quality job)
- Replace o1-mini with o1 in test_reasoning_tokens_no_price_set (model removed
from cost map)
- Replace gemini-2.5-pro-exp-03-25 with gemini-2.5-pro in
test_generic_cost_per_token_above_200k_tokens (model removed from cost map)
- Fix test_get_cost_for_anthropic_web_search to use claude-3-7-sonnet-20250219
with custom_llm_provider='anthropic' so web search cost is computed correctly
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
- Mock litellm.completion_cost in test_pass_through_success_handler_gemini_routing
to decouple it from model_prices_and_context_window.json; prevents the same
breakage if gemini-2.0-flash is ever removed from the pricing map
- Revert _create_passthrough_logging_payload URL back to gemini-1.5-flash to
eliminate inconsistency with the other tests that use gemini-1.5-flash explicitly
* fix(test): null AWS SigV4 fields on MagicMock in test_inherit_credentials_from_existing_server
* fix(test): null AWS SigV4 fields on MagicMock in test_add_session_mcp_server_caches_and_redacts_credentials
* fixed mcp api
* added non-admin test
* resolved greptile comemnt
* fix: add IP filtering to get_mcp_server_by_id path in fetch_mcp_server
Apply _is_server_accessible_from_ip check after get_mcp_server_by_id lookup
to prevent external callers from accessing MCP servers configured with
available_on_public_internet=False when they know the server_id.
Made-with: Cursor
- Remove dead code: is_expired was assigned but never used in
mcp_management_endpoints.py (the raw expires_at timestamp is passed
directly to the client per existing comment)
- Handle Azure DALL-E 3 ModelDeprecated (HTTP 410) error gracefully in
base_image_generation_test.py so CI doesn't fail on deprecated model
deployments
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
gemini/gemini-2.5-flash lacks cache_creation_input_token_cost in the
model cost map, causing a TypeError when the test multiplies
cache_creation_input_tokens by None. Use claude-haiku-4-5 instead,
which has the required prompt caching cost fields.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(mcp): fix OpenAPI OAuth flow — transport mapping, error messages, and discovery bypass
Three bugs fixed to make the end-to-end OAuth flow work for OpenAPI MCP servers:
1. **Transport mapping in getTemporaryPayload**: `TRANSPORT.OPENAPI` is a UI-only concept;
the backend only accepts `"http"`, `"sse"`, or `"stdio"`. The pre-OAuth temp-session
call was sending `transport: "openapi"` and getting a 422. Fixed by mapping to `"http"`.
2. **deriveErrorMessage handles FastAPI 422 arrays**: FastAPI validation errors return
`detail` as an array of `{loc, msg, type}` objects. The shared error extractor was
returning the array directly, causing `Error: [object Object]`. Fixed to map each
item to its `.msg` field.
3. **Skip OAuth discovery when authorization_url already provided**: `build_mcp_server_from_table`
was unconditionally calling `_descovery_metadata(server_url)` for OAuth servers. For
OpenAPI servers the url is the spec JSON file, not the API base — this caused a timeout
fetching e.g. the GitHub spec (2 MB). Fixed by skipping discovery when `authorization_url`
is already set.
Also: collapsible auth section in MCP server form, "Create OAuth App →" link next to
Client ID when a docs URL is available (e.g. GitHub OAuth App creation page), and
`extractErrorMessage` helper in `useMcpOAuthFlow` for cleaner error display.
* refactor(mcp): extract needs_discovery flag and reduceStaticHeaders helper
* feat(mcp): user OAuth connect flow — OAuthConnectModal, MCPCredentialsTab, useUserMcpOAuthFlow
Adds the user-facing MCP OAuth2 PKCE connect flow:
- OAuthConnectModal: modal that launches the PKCE flow for a user to connect to an MCP server
- MCPCredentialsTab: credentials management tab in the MCP apps panel
- useUserMcpOAuthFlow: hook that handles the full PKCE auth code exchange for user-level connections
- MCPAppsPanel: wires up the new credentials tab and connect modal
- ChatPage: further cleanup after responses-API revert
- db.py / mcp_management_endpoints.py / _types.py: backend support for storing user MCP credentials
* fix(mcp): make client_id optional in /authorize — use server's stored client_id when not provided
* address greptile review feedback
* fix(mcp): narrow bare except to RecordNotFoundError in BYOK credential delete
* refactor(mcp): move inline imports to module level in db.py
* docs(claude): add MCP OAuth, transport mapping, and browser storage patterns
* fix(security): remove accessToken from sessionStorage in OAuth flow state
The LiteLLM API key was being serialised into sessionStorage as part of
StoredFlowState. After the OAuth redirect the component re-mounts with the
same accessToken prop, so it never needed to be stored. Read it from props
in resumeOAuthFlow instead.
* fix(ui): remove duplicate extractErrorMessage, sessionStorage-only in admin OAuth hook, call delete API on disconnect
* fix(ui): guard resumeOAuthFlow against wrong hook instance consuming OAuth result
* fix(ui): separate OAuth result keys per flow, sessionStorage-only, surface revoke errors
* fix(ui): remove dead OAuthConnectModal, revert tsconfig jsx mode to preserve
* fix(mcp): guard BYOK overwrite in oauth credential store, raise clear error when client_id absent
* fix: forward OAuth error params in callback, fix BYOK guard exception handling in db.py
- MCP tests: set mock_mcp_server.oauth2_flow = None to prevent MagicMock
leaking into Pydantic Literal validation for MCPServer
- AgentCore tests: pass api_key="test-jwt-token" to bypass SigV4 credential
lookup that fails in CI without AWS credentials
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use group_by instead of find_many(distinct) in /tag/list to avoid full table scan
The list_tags endpoint used Prisma's find_many(distinct=["tag"]) to discover
dynamic tags from LiteLLM_DailyTagSpend. Prisma's distinct is a client-side
post-processing filter that fetches all rows with all columns before
deduplicating, causing a full table scan of 3M+ rows on every usage page load.
Replace with Prisma's group_by() which generates proper GROUP BY SQL, letting
the database handle deduplication efficiently. NULL tags are filtered out via
where={"tag": {"not": None}}, and created_at/updated_at are preserved via
_min/_max aggregates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review feedback
- Remove unused LiteLLM_DailyTagSpendTable class and datetime import
- Simplify created_at/updated_at access (values guaranteed by spend system)
- Remove DB call assertions from tests, keep response-only assertions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: RheagalFire <arishalam121@gmail.com>
* fix: batch key queries in list_team to eliminate N+1
The list_team endpoint issued one DB query per team to fetch keys,
causing N+1 query overhead. Replace with a single batched IN query.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review feedback on team list test
- Rename test to include "v1" for consistency with other test names
- Remove unnecessary DB call assertion, keep behavioral assertions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: RheagalFire <arishalam121@gmail.com>
Previously this setting was only honored via the AUTO_REDIRECT_UI_LOGIN_TO_SSO
environment variable. Now it can also be set in config.yaml under general_settings,
matching the pattern used by other proxy settings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two fixes based on PR feedback:
1. Move callback dispatch before the prisma_client check so audit logs
still reach S3/Datadog even if the DB is down. Also changed the
prisma_client=None case from raising an exception to logging an error
and returning gracefully.
2. Attach a done_callback to asyncio tasks created for audit log
callbacks so exceptions are logged through verbose_proxy_logger
instead of silently swallowed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add verbose_logger.warning when user-specified region is overridden by
supported_regions. Remove now-unused is_global_only_vertex_model function
and its tests since get_vertex_region handles all region logic directly.
When a request arrives without an API key, auth fails and the failure
hook triggers spend tracking. _is_master_key was called with api_key=None,
causing secrets.compare_digest to raise TypeError. Add a None guard for
api_key matching the existing guard for _master_key.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- get_vertex_region now overrides user-specified region when it's not in
the model's supported_regions list (prevents 404 for users with a
global VERTEXAI_LOCATION default hitting global-only models)
- Add supported_regions: ["global"] to glm-5-maas in both JSON files
- Update tests to cover the override behavior
Replace removed deprecated models (claude-3-5-sonnet-20241022,
claude-3-5-haiku-20241022, claude-3-5-haiku-latest) with current
models in web_search and cost calculation tests.
- Remove redundant get_vertex_region() call in partner models main.py
(already called inside get_complete_vertex_url)
- Rewrite test mocks to use patch.dict(litellm.model_cost) instead of
patching the removed is_global_only_vertex_model symbol
- Align test assertions with new behavior: user-specified region is
preserved (not overridden) for global-only models
Address review comments from PR #23325:
- Remove duplicate `from unittest.mock import MagicMock, patch` and unused `json` import
- Parameterize HTTP method in `_make_mock_response` helper so mock requests
use the correct method (GET/POST/DELETE) matching each test scenario
* Replace broken Anthropic Skills API integration test with unit tests
* Update tests/test_litellm/test_anthropic_skills_transformation.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Aarish Alam <arishalam121@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Implement Anthropic Files API (upload, retrieve, list, delete, content)
using the BaseFilesConfig provider pattern. Adds multipart form-data
support to BaseLLMHTTPHandler for file uploads.
