Tests used call_policy throughout but the actual API model uses
input_policy and output_policy. Updated _make_tool_row helper,
list filter query param, and policy update request/response assertions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The _resolve_model_for_cost_lookup function was only checking
litellm_params.model when resolving model names from the router.
For Azure custom deployment names (e.g. azure/openai/gpt-5.3-codex),
this deployment name doesn't exist in the model cost map, so cost
returned /bin/zsh.
Now checks model_info.base_model and litellm_params.base_model first,
falling back to litellm_params.model only if no base_model is set.
This matches how the router resolves base_model everywhere else.
Add filtering capabilities to /key/list endpoint for project_id and access_group_id parameters. Both filters work globally across all visibility rules and stack with existing sort/pagination params. Added comprehensive unit tests for the new filters.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
When a client sends null for assigned_team_ids or assigned_key_ids, ensure
the DB receives [] instead of null, preventing null from being stored where
empty list is expected. Extend test to verify the DB call uses [].
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
set(None) raises TypeError when a client sends null for assigned_team_ids or
assigned_key_ids. Add `or []` to handle null safely, consistent with create.
Add test covering this case.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When creating, updating, or deleting access groups, automatically keep
team and key access_group_ids in sync with the access group's assigned_team_ids
and assigned_key_ids. Includes transaction-based DB updates, cache patching,
and handles out-of-sync data by unioning assigned_* fields with hasSome queries.
Adds 12 new tests covering sync behavior across all three CRUD operations.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace Prisma ORM count/find_many calls with two query_raw calls that
only project the key_alias column. The Prisma client wrapper does not
support SELECT projection via find_many, so raw SQL is used to keep
memory usage proportional to the page size rather than total key count.
Update tests to mock query_raw instead of count/find_many.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
LiteLLM_VerificationTokenActions.find_many() does not support the
select keyword argument. Remove it and drop the corresponding test.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Add select={"key_alias": True} to the find_many call so only the alias
column is fetched from the database instead of full token rows. Add
five unit tests in test_key_management_endpoints.py covering response
shape, pagination skip/take computation, search filter injection,
absence of contains filter when no search term is given, and the
select-only-alias optimization.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* feat(agents): assign virtual keys to agents
- Add agent_id field to LiteLLM_VerificationToken (schema.prisma + _types.py)
- Pass agent_id through key generation endpoint so keys can be scoped to an agent
- Refactor Add Agent wizard to 3-step flow (Configure → Assign Key → Ready)
- Configure: all agent fields, custom/other type with just name+description
- Assign Key: create new key or reassign existing key to agent
- URL is now optional for easy discovery
- Add "Agent" ownership option to Create Key modal on Virtual Keys page
with agent selector dropdown
- Extract CreatedKeyDisplay into shared component, reused in both flows
- Add keyCreateForAgentCall networking helper
- Add test for agent_id key generation
* fix(agents): code quality fixes from self-review
- Fix test_generate_key_helper_fn_agent_id: remove bare except clause,
use explicit assert mock_insert.called, use .kwargs for clean arg access
- Remove no-op conditional in handleNext (both branches were identical)
- Validate selectedExistingKey before calling keyUpdateCall
- Validate selectedAgentId before setting on formValues in create_key_button
* fix(ui): replace deprecated Tremor Button with Ant Design Button in CreatedKeyDisplay
* Add Ask AI chat component to Usage page
- Create UsageAIChatModal component with streaming chat interface
- Integrate with existing model hub for model selection
- Pass usage data context (spend, models, providers, keys) to AI
- Add Ask AI button next to Export Data button in global view
- Add tests for the new component and integration
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Convert Ask AI from modal to right-side sliding panel
- Replace UsageAIChatModal with UsageAIChatPanel
- Panel slides in from right side, usage page stays visible
- Full-height panel with header, model selector, chat area, and input
- Smooth CSS transition for open/close animation
- Update tests for new panel component (34 tests passing)
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Remove build output directory from tracking
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Add backend AI usage chat endpoint with tool calling
Backend:
- New /usage/ai/chat SSE streaming endpoint
- AI agent has get_usage_data tool that queries /user/daily/activity/aggregated
- Follows same architecture as policy AI suggest (litellm.acompletion + tools)
- Non-admin users are restricted to their own data
- 12 backend unit tests
Frontend:
- Panel now calls /usage/ai/chat backend endpoint via SSE
- Removed direct OpenAI client calls from frontend
- Added usageAiChatStream networking function following enrichPolicyTemplateStream pattern
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Make model selection optional, default to gpt-4o-mini on backend
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Add team/tag tools, status indicators, and improved AI agent
- AI agent now has 3 tools: get_usage_data, get_team_usage_data, get_tag_usage_data
- Stream status events (Thinking... Fetching... Analyzing...) to UI
- Frontend shows spinner + status text during tool execution
- Better system prompt guiding tool selection
- Entity summariser for team/tag data with ranked breakdowns
- 13 backend tests, 34 frontend tests passing
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Fix: inject today's date into system prompt so AI resolves relative dates correctly
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Show tool calls as distinct steps + render markdown in responses
- Backend emits tool_call events with tool_name, label, args, and status
- Frontend shows each tool call as a step with ✓/spinner/✗ indicator
- Tool call steps show icon, label, date range, and filters
- AI responses rendered with ReactMarkdown (bold, lists, tables, code)
- Cursor-like UX: Thinking → tool calls → Analyzing → streamed answer
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Refactor backend for code quality: proper types, constants, all functions ≤50 LOC
- TypedDict for SSE events (SSEStatusEvent, SSEToolCallEvent, etc.) and ToolHandler
- Constants for table names, entity fields, temperature, page sizes, top-N limits
- Shared _query_activity() eliminates duplicated fetch logic
- _accumulate_breakdown() + _ranked_lines() replace inline aggregation loops
- Extracted _process_tool_call() and _stream_final_response() from main stream fn
- Black + Ruff clean, all 15 functions verified ≤50 LOC
- Replaced Tremor Button with Antd Button in panel (Tremor deprecated per AGENTS.md)
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Address greptile review: security fixes and input validation
- Restrict team/tag tools to admin-only users (non-admins only get get_usage_data)
- Constrain ChatMessage.role to Literal['user', 'assistant'] to prevent system prompt injection
- Add test for base tools restriction (non-admin gets 1 tool, admin gets 3)
- Issues 3 (unused imports) and 4 (inline datetime) were already fixed in prior commit
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Address greptile round 2: sanitize errors, defense-in-depth allowlist, revert tsconfig
- Sanitize error messages: generic 'An internal error occurred' sent to client,
full exception logged server-side via verbose_proxy_logger
- Defense-in-depth: _process_tool_call validates fn_name against role-based
allowlist before dispatch (even though LLM only receives allowed tools)
- Revert tsconfig.json jsx back to 'preserve' (Next.js recommended default)
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Role-scoped system prompt + additional test coverage
- System prompt is now role-aware: admin sees all 3 tool descriptions,
non-admin only sees get_usage_data (consistent with tool filtering)
- Added tests: non-admin prompt excludes team/tag tools, date injection
- 15 backend tests, 34 frontend tests passing
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Fix LLM arg validation + cap conversation size at 20 messages
- _resolve_fetch_kwargs uses .get() with ValueError for missing dates
(handles malformed LLM tool arguments gracefully)
- MAX_CHAT_MESSAGES = 20 constant; backend truncates to last 20
- Frontend also sends only last 20 messages per request
- Prevents excessive token usage and context-length errors
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* feat(proxy): tool policies - auto-discover tools, manage policies, guardrail enforcement
- New LiteLLM_ToolTable in schema.prisma to store discovered tools
- Auto-discovery: tools seen in LLM responses get upserted via ToolDiscoveryQueue
(hooks into DBSpendUpdateWriter, same pipeline as spend tracking)
- Management endpoints: GET /v1/tool/list, GET /v1/tool/{name}, POST /v1/tool/policy
- ToolPolicyGuardrail: blocks tool_calls in responses based on policy setting
- UI: Tool Policies page under Guardrails section with policy selector,
filters by policy/team/key, live tail, sortable table
- Unit tests for queue, writer, endpoints, guardrail
* feat(tool-policies): track call_count + discover tools from request body and /messages API
- Add call_count column to LiteLLM_ToolTable; incremented on every flush
- Extract tools from request body too (not just response tool_calls):
- OpenAI /chat/completions: tools[].function.name
- Anthropic /messages pass-through: request_body.tools[].name
- Show call_count column in UI table (sortable)
- UI: drop dual_llm option, keep only trusted/blocked
* fix: address greptile review feedback
- Remove redundant @@index([tool_name]) from schema.prisma (tool_name has @unique which already creates an index)
- Replace gen_random_uuid()::text with str(uuid.uuid4()) for portability
- Rewrite test_tool_registry_writer.py to mock execute_raw/query_raw (actual implementation) instead of Prisma model methods
- Fix test patches in test_tool_management_endpoints.py to target source modules since imports are inside function bodies
- Add "Tool Policies" page title to ToolPolicies.tsx
* fix: address greptile review round 2
- Replace NOW() with Python datetime parameter in tool_registry_writer (SQLite portability)
- Fix cache key collision in tool_policy_guardrail: use null-byte separator instead of colon
- Remove type==function filter from request-side tool extraction to match response-side behavior
- Clear seen_tool_names on flush so call_count increments per batch cycle not per pod lifetime
* fix: address greptile review round 3
- Fix test_seen_names_persist_across_flushes to match actual per-flush-cycle behavior
- Update module docstring in tool_discovery_queue.py to accurately describe flush behavior
- Add created_at/updated_at to raw SQL INSERT in batch_upsert_tools and update_tool_policy
* fix: cache tool policies per tool name not per combination
Previously the cache key was built from the full set of tool names in a
request, so each unique combination of tools got its own cold cache entry
and triggered a separate DB query. With N distinct tools across requests
this was effectively a DB hit on every request.
Now each tool name is cached individually. Cache hits are checked per
tool, only missing tools are fetched from DB in a single batch query,
and each result is cached separately. Once a tool's policy is warm,
any subsequent request using that tool benefits from the cache regardless
of what other tools are in the request.
* Update ui/litellm-dashboard/src/components/ToolPolicies.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
POST /access_group/new and PUT /access_group/{name}/update now accept an
optional model_ids list that targets specific deployments by their unique
model_id, instead of tagging every deployment that shares a model_name.
When model_ids is provided it takes priority over model_names, giving
API callers the same single-deployment precision that the UI already has
via PATCH /model/{model_id}/update.
Backward compatible: model_names continues to work as before.
Closes#21544
Add user_email field to LiteLLM_OrganizationMembershipTable with a model_validator
that populates it from the nested user object, and update the /organization/info
Prisma query to select user_email from the related user record.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Regular team members could not see service accounts belonging to their
team. Additionally, former team members could still see service accounts
they created via the created_by filter after leaving the team.
- Add get_member_team_ids() to retrieve all teams a user belongs to
- Regular members now see team service accounts (user_id=NULL) but not
other members' personal keys
- Scope created_by filter to the user's current team memberships
- Add 7 unit tests covering the new visibility rules
Replace find_many + Python-side aggregation with a single SQL GROUP BY
query via query_raw in get_daily_activity_aggregated. This collapses
rows across entities (users/teams/orgs) in the database, reducing ~150k
rows to ~2-3k grouped rows before transfer to Python.
Also adds composite indexes (entity_id, date) to all 6 daily spend
tables for faster filtered queries.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The admin bulk-delete path (prisma_client.delete_data) was never comparing
the DB return value against the requested tokens, so failed_tokens was
always empty for admins even when the DB silently skipped some tokens.
Adds the same mismatch check as the non-admin path, and a new test that
exercises admin bulk-delete returning fewer tokens than requested.
Fixes gap identified by Greptile review on #21609.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Instead of raising an exception when some requested tokens are not found
in the DB, populate `failed_tokens` in the return dict so callers can
inspect which keys were not deleted without catching a generic Exception.
Supersedes #12577.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix aviation safety topic filter: remove overly broad exceptions, add cockpit access block words
* fix airline brand protection filter: add identifier words, competitor/ops block words, tighten exceptions
* add example_sentences to all policy templates + topic-filtering and prompt-injection templates
* add policy_endpoints package with AI policy suggester
* update test patch targets for policy_endpoints package move
* add unit tests for AI policy suggester
* add suggestPolicyTemplates networking function
* add AI suggestion modal component
* add Use AI button and template loading callback to PolicyTemplates
* wire up AI suggestion modal in policies page
* fix policy_templates_backup.json path after package move
* add estimated_latency field to all policy templates
* use llm_router and accept model parameter in ai_policy_suggester
* add model param to suggest templates endpoint
* pass model param in suggestPolicyTemplates
* polish ai suggestion modal: model selector, auto-growing textareas, latency badges
* add template queue for processing multiple AI-suggested templates
* show template progress badge in guardrail selection modal
* fix(scim): handle deprovisioning operations without path field
When SCIM providers send deprovisioning requests without a path field
(e.g., {"op": "replace", "value": {"active": false}}), the code was
storing the value under an empty string key in metadata.
This fix:
- Detects operations with no path where value is a dict
- Extracts and handles known fields like 'active' correctly
- Sets metadata["scim_active"] = false instead of metadata[""] = {"active": false}
Fixes: SCIM deprovisioning creating empty string keys in user metadata
* fix(scim): handle all known fields in operations without path
Extended the fix to handle all SCIM fields (not just active) when
operations have no path field:
- active -> scim_active
- displayName -> user_alias
- externalId -> user_id
- name.givenName/familyName -> scim_metadata
Added comprehensive test for multiple fields without path.
Addresses Greptile review feedback on RFC 7644 compliance.
* trigger PR update
* fix aviation safety topic filter: remove overly broad exceptions, add cockpit access block words
* fix airline brand protection filter: add identifier words, competitor/ops block words, tighten exceptions
* update policy templates with competitor pre/post guardrails and streaming enrichment
* sync policy_templates_backup.json with policy_templates.json
* add streaming enrichment endpoint, competitor variations, and model selection
* add streaming enrichPolicyTemplate networking function
* rewrite template parameter modal with streaming tags, AI/manual toggle, model selector
* update policies panel to pass enrichment options and show discovered competitors
* show discovered competitors as tags in guardrail selection modal
* use llm_router instead of litellm.acompletion, extract helpers, move constants
* validate competitors list size, cap variation prompt input
* add refinement instruction support for competitor discovery
* add instruction and existingCompetitors params to streaming enrichment
* add refinement input for iterating on competitor list with AI
* emit status events during variation generation so UI shows progress
* add onStatus callback to streaming enrichment
* show status spinner during variation generation, widen modal to 700px
* add tests for competitor enrichment helper functions
Add a new team member permission `/team/daily/activity` that allows
non-admin team members to see all team usage data, not just their own.
## Changes
- Add `TEAM_DAILY_ACTIVITY` to `KeyManagementRoutes` enum and available
team member permissions
- Add `_team_member_has_permission` helper in `common_utils.py`
- Modify `/team/daily/activity` endpoint to skip API key filtering when
the member has this permission
- Add permission description and method detection in the UI
- Add backend and frontend tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(ui/): initial commit adding a compliance testing playground
allow proxy admins to test policies and guardrails against datasets
* feat(ui/): make score more friendly
* feat(policy_endpoints.py): new helper function for testing policies
* feat(policy_endpoints.py): expose new endpoint for testing policies and guardrails
enables compliance playground to work as expected
* feat(complianceui.tsx): show returned text
When `default_internal_user_params` was set, `insert_sso_user()` only
preserved SSO-provided roles if `role_mappings` was explicitly configured.
Roles from other valid SSO sources (Microsoft app_roles,
GENERIC_USER_ROLE_ATTRIBUTE, custom SSO handlers) were silently
overwritten with the default "internal_user" role, causing admin users
to be downgraded on first login or after user deletion.
Replace the `role_mappings_configured` gate with `_should_use_role_from_sso_response()`
which validates the role is a recognized LitellmUserRoles value regardless
of origin. Also removes an unnecessary DB round-trip to litellm_ssoconfig
on every new SSO user creation.
Fixes: admin users seeing internal-user UI after SSO login
Fixes: test_get_redirect_url_for_sso flaking due to local env vars
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Four finally blocks in test_internal_user_endpoints.py and one in
test_ui_sso.py used the pattern:
if original_default_params is not None:
litellm.default_internal_user_params = original_default_params
else:
delattr(litellm, "default_internal_user_params")
Since the attribute is defined in litellm/__init__.py with a default of
None, `getattr(litellm, "default_internal_user_params", None)` returns
None. The else branch then calls delattr(), permanently removing the
attribute from the module for the rest of the process.
Subsequent tests in the same pytest-xdist worker (e.g.
test_add_new_member_* in test_management_helpers_utils.py) then fail
with: AttributeError: module 'litellm' has no attribute
'default_internal_user_params'
Fix: replace all five flawed finally blocks with a simple assignment:
litellm.default_internal_user_params = original_default_params
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test_role_mappings_override_default_internal_user_params was calling
delattr(litellm, 'default_internal_user_params') in its finally block
when the original value was None. This removes the attribute entirely from
the module, causing subsequent tests in the same xdist worker to get
AttributeError when accessing litellm.default_internal_user_params (because
litellm.__getattr__ has no handler for this name).
Fix: always restore the attribute by assignment (litellm.default_internal_user_params = original_default_params)
rather than deleting it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Default user_id to caller's own ID for non-admins instead of 403 when
omitted, preserving backward compatibility for API consumers
- Apply same fix to aggregated endpoint
- Update test to verify defaulting behavior instead of expecting 403
- Add useEffect to sync selectedUserId when auth state settles in
UsagePageView to handle async auth initialization
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix HTTPException swallowed by broad except block in get_user_daily_activity
and get_user_daily_activity_aggregated: re-raise HTTPException before the
generic handler so 403 status codes propagate correctly
- Add status_code assertions in non-admin access tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: SSO PKCE support fails in multi-pod Kubernetes deployments
* fix: virutal key grace period from env/UI
* fix: refactor, race condition handle, fstring sql injection
* fix: add async call to avoid server pauses
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: add await in tests
* add modify test to perform async run
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix grace period with better error handling on frontend and as per best practices
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: as per request changes
* Update litellm/proxy/utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Fix errors when callbacks are invoked for file delete operations:
* Fix errors when callbacks are invoked for file operations
* Fix: pass deployment credentials to afile_retrieve in managed_files post-call hook
* Fix: bypass managed files access check in batch polling by calling afile_content directly
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: afile_retrieve returns unified ID for batch output files
* fix: batch retrieve returns unified input_file_id
* fix(chatgpt): drop unsupported responses params for Codex
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(chatgpt): ensure Codex request filters unsupported params
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix deleted managed files returning 403 instead of 404
* Add comments
* Update litellm/proxy/utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: thread deployment model_info through batch cost calculation
batch_cost_calculator only checked the global cost map, ignoring
deployment-level custom pricing (input_cost_per_token_batches etc.).
Add optional model_info param through the batch cost chain and pass
it from CheckBatchCost.
* fix(deps): add pytest-postgresql for db schema migration tests
The test_db_schema_migration.py test requires pytest-postgresql but it was
missing from dependencies, causing import errors:
ModuleNotFoundError: No module named 'pytest_postgresql'
Added pytest-postgresql ^6.0.0 to dev dependencies to fix test collection
errors in proxy_unit_tests.
This is a pre-existing issue, not related to PR #21277.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): replace caplog with custom handler for parallel execution
The cost calculation log level tests were failing when run with pytest-xdist
parallel execution because caplog doesn't work reliably across worker processes.
This causes "ValueError: I/O operation on closed file" errors.
Solution: Replace caplog fixture with a custom LogRecordHandler that directly
attaches to the logger. This approach works correctly in parallel execution
because each worker process has its own handler instance.
Fixes test failures in PR #21277 when running with --dist=loadscope.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): correct async mock for video generation logging test
The test was failing with AuthenticationError because the mock wasn't
intercepting the actual HTTP handler calls. This caused real API calls
with no API key, resulting in 401 errors.
Root cause: The test was patching the wrong target using string path
'litellm.videos.main.base_llm_http_handler' instead of using patch.object
on the actual handler instance. Additionally, it was mocking the sync
method instead of async_video_generation_handler.
Solution: Use patch.object with side_effect pattern on the correct
async handler method, following the same pattern used in
test_video_generation_async().
Fixes test failure in PR #21277 when running with --dist=loadscope.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): add cleanup fixture and no_parallel mark for MCP tests
Two MCP server tests were failing when run with pytest-xdist parallel
execution (--dist=loadscope):
- test_mcp_routing_with_conflicting_alias_and_group_name
- test_oauth2_headers_passed_to_mcp_client
Both tests showed assertion failures where mocks weren't being called
(0 times instead of expected 1 time).
Root cause: These tests rely on global_mcp_server_manager singleton
state and complex async mocking that doesn't work reliably with
parallel execution. Each worker process can have different state
and patches may not apply correctly.
Solution:
1. Added autouse fixture to clean up global_mcp_server_manager registry
before and after each test for better isolation
2. Added @pytest.mark.no_parallel to these specific tests to ensure
they run sequentially, avoiding parallel execution issues
This approach maintains test reliability while allowing other tests
in the file to still benefit from parallelization.
Fixes test failures exposed by PR #21277.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Regenerate poetry.lock with Poetry 2.3.2
Updated lock file to use Poetry 2.3.2 (matching main branch standard).
This addresses Greptile feedback about Poetry version mismatch.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Remove unused pytest import and add trailing newline
- Removed unused pytest import (caplog fixture was removed)
- Added missing trailing newline at end of file
Addresses Greptile feedback (minor style issues).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Remove redundant import inside test method
The module litellm.videos.main is already imported at the top of
the file (line 21), so the import inside the test method is redundant.
Addresses Greptile feedback (minor style issue).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Fix converse anthropic usage object according to v1/messages specs
* Add routing based on if reasoning is supported or not
* add fireworks_ai/accounts/fireworks/models/kimi-k2p5 in model map
* Removed stray .md file
* fix(bedrock): clamp thinking.budget_tokens to minimum 1024
Bedrock rejects thinking.budget_tokens values below 1024 with a 400
error. This adds automatic clamping in the LiteLLM transformation
layer so callers (e.g. router with reasoning_effort="low") don't
need to know about the provider-specific minimum.
Fixes#21297
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: improve Langfuse test isolation to prevent flaky failures (#21093)
The test was creating fresh mocks but not fully isolating from setUp state,
causing intermittent CI failures with 'Expected generation to be called once.
Called 0 times.'
Instead of creating fresh mocks, properly reset the existing setUp mocks to
ensure clean state while maintaining proper mock chain configuration.
* feat(s3): add support for virtual-hosted-style URLs (#21094)
Add s3_use_virtual_hosted_style parameter to support AWS S3 virtual-hosted-style URL format (bucket.endpoint/key) alongside the existing path-style format (endpoint/bucket/key).
This enables compatibility with S3-compatible services like MinIO and aligns with AWS S3 official terminology.
* Addressed greptile comments to extract common helpers and return 404
* Allow effort="max" for Claude Opus 4.6 (#21112)
* fix(aiohttp): prevent closing shared ClientSession in AiohttpTransport (#21117)
When a shared ClientSession is passed to LiteLLMAiohttpTransport,
calling aclose() on the transport would close the shared session,
breaking other clients still using it.
Add owns_session parameter (default True for backwards compatibility)
to AiohttpTransport and LiteLLMAiohttpTransport. When a shared session
is provided in http_handler.py, owns_session=False is set to prevent
the transport from closing a session it does not own.
This aligns AiohttpTransport with the ownership pattern already used
in AiohttpHandler (aiohttp_handler.py).
* perf(spend): avoid duplicate daily agent transaction computation (#21187)
* fix: proxy/batches_endpoints/endpoints.py:309:11: PLR0915 Too many statements (54 > 50)
* fix mypy
* Add doc for OpenAI Agents SDK with LiteLLM
* Add doc for OpenAI Agents SDK with LiteLLM
* Update docs/my-website/sidebars.js
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix mypy
* Update tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add blog fffor Managing Anthropic Beta Headers
* Add blog fffor Managing Anthropic Beta Headers
* correct the time
* Fix: Exclude tool params for models without function calling support (#21125) (#21244)
* Fix tool params reported as supported for models without function calling (#21125)
JSON-configured providers (e.g. PublicAI) inherited all OpenAI params
including tools, tool_choice, function_call, and functions — even for
models that don't support function calling. This caused an inconsistency
where get_supported_openai_params included "tools" but
supports_function_calling returned False.
The fix checks supports_function_calling in the dynamic config's
get_supported_openai_params and removes tool-related params when the
model doesn't support it. Follows the same pattern used by OVHCloud
and Fireworks AI providers.
* Style: move verbose_logger to module-level import, remove redundant try/except
Address review feedback from Greptile bot:
- Move verbose_logger import to top-level (matches project convention)
- Remove redundant try/except around supports_function_calling() since it
already handles exceptions internally via _supports_factory()
* fix(index.md): cleanup str
* fix(proxy): handle missing DATABASE_URL in append_query_params (#21239)
* fix: handle missing database url in append_query_params
* Update litellm/proxy/proxy_cli.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(mcp): revert StreamableHTTPSessionManager to stateless mode (#21323)
PR #19809 changed stateless=True to stateless=False to enable progress
notifications for MCP tool calls. This caused the mcp library to enforce
mcp-session-id headers on all non-initialize requests, breaking MCP
Inspector, curl, and any client without automatic session management.
Revert to stateless=True to restore compatibility with all MCP clients.
The progress notification code already handles missing sessions gracefully
(defensive checks + try/except), so no other changes are needed.
Fixes#20242
* UI - Content Filters, help edit/view categories and 1-click add categories + go to next page (#21223)
* feat(ui/): allow viewing content filter categories on guardrail info
* fix(add_guardrail_form.tsx): add validation check to prevent adding empty content filter guardrails
* feat(ui/): improve ux around adding new content filter categories
easy to skip adding a category, so make it a 1-click thing
* Fix OCI Grok output pricing (#21329)
* fix(proxy): fix master key rotation Prisma validation errors
_rotate_master_key() used jsonify_object() which converts Python dicts
to JSON strings. Prisma's Python client rejects strings for Json-typed
fields — it requires prisma.Json() wrappers or native dicts.
This affected three code paths:
- Model table (create_many): litellm_params and model_info converted to
strings, plus created_at/updated_at were None (non-nullable DateTime)
- Config table (update): param_value converted to string
- Credentials table (update): credential_values/credential_info
converted to strings
Fix: replace jsonify_object() with model_dump(exclude_none=True) +
prisma.Json() wrappers for all Json fields. Wrap model delete+insert
in a Prisma transaction for atomicity. Add try/except around MCP
server rotation to prevent non-critical failures from blocking the
entire rotation.
---------
Co-authored-by: Harshit Jain <harshitjain0562@gmail.com>
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Ephrim Stanley <ephrim.stanley@point72.com>
Co-authored-by: Jay Prajapati <79649559+jayy-77@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Julio Quinteros Pro <jquinter@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: mjkam <mjkam@naver.com>
Co-authored-by: Fly <48186978+tuzkiyoung@users.noreply.github.com>
Co-authored-by: Kristoffer Arlind <13228507+KristofferArlind@users.noreply.github.com>
Co-authored-by: Constantine <Runixer@gmail.com>
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: Atharva Jaiswal <92455570+AtharvaJaiswal005@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
