* fix: SSO PKCE support fails in multi-pod Kubernetes deployments
* fix: virutal key grace period from env/UI
* fix: refactor, race condition handle, fstring sql injection
* fix: add async call to avoid server pauses
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: add await in tests
* add modify test to perform async run
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix grace period with better error handling on frontend and as per best practices
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: as per request changes
* Update litellm/proxy/utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Fix errors when callbacks are invoked for file delete operations:
* Fix errors when callbacks are invoked for file operations
* Fix: pass deployment credentials to afile_retrieve in managed_files post-call hook
* Fix: bypass managed files access check in batch polling by calling afile_content directly
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: afile_retrieve returns unified ID for batch output files
* fix: batch retrieve returns unified input_file_id
* fix(chatgpt): drop unsupported responses params for Codex
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(chatgpt): ensure Codex request filters unsupported params
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix deleted managed files returning 403 instead of 404
* Add comments
* Update litellm/proxy/utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: thread deployment model_info through batch cost calculation
batch_cost_calculator only checked the global cost map, ignoring
deployment-level custom pricing (input_cost_per_token_batches etc.).
Add optional model_info param through the batch cost chain and pass
it from CheckBatchCost.
* fix(deps): add pytest-postgresql for db schema migration tests
The test_db_schema_migration.py test requires pytest-postgresql but it was
missing from dependencies, causing import errors:
ModuleNotFoundError: No module named 'pytest_postgresql'
Added pytest-postgresql ^6.0.0 to dev dependencies to fix test collection
errors in proxy_unit_tests.
This is a pre-existing issue, not related to PR #21277.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): replace caplog with custom handler for parallel execution
The cost calculation log level tests were failing when run with pytest-xdist
parallel execution because caplog doesn't work reliably across worker processes.
This causes "ValueError: I/O operation on closed file" errors.
Solution: Replace caplog fixture with a custom LogRecordHandler that directly
attaches to the logger. This approach works correctly in parallel execution
because each worker process has its own handler instance.
Fixes test failures in PR #21277 when running with --dist=loadscope.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): correct async mock for video generation logging test
The test was failing with AuthenticationError because the mock wasn't
intercepting the actual HTTP handler calls. This caused real API calls
with no API key, resulting in 401 errors.
Root cause: The test was patching the wrong target using string path
'litellm.videos.main.base_llm_http_handler' instead of using patch.object
on the actual handler instance. Additionally, it was mocking the sync
method instead of async_video_generation_handler.
Solution: Use patch.object with side_effect pattern on the correct
async handler method, following the same pattern used in
test_video_generation_async().
Fixes test failure in PR #21277 when running with --dist=loadscope.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): add cleanup fixture and no_parallel mark for MCP tests
Two MCP server tests were failing when run with pytest-xdist parallel
execution (--dist=loadscope):
- test_mcp_routing_with_conflicting_alias_and_group_name
- test_oauth2_headers_passed_to_mcp_client
Both tests showed assertion failures where mocks weren't being called
(0 times instead of expected 1 time).
Root cause: These tests rely on global_mcp_server_manager singleton
state and complex async mocking that doesn't work reliably with
parallel execution. Each worker process can have different state
and patches may not apply correctly.
Solution:
1. Added autouse fixture to clean up global_mcp_server_manager registry
before and after each test for better isolation
2. Added @pytest.mark.no_parallel to these specific tests to ensure
they run sequentially, avoiding parallel execution issues
This approach maintains test reliability while allowing other tests
in the file to still benefit from parallelization.
Fixes test failures exposed by PR #21277.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Regenerate poetry.lock with Poetry 2.3.2
Updated lock file to use Poetry 2.3.2 (matching main branch standard).
This addresses Greptile feedback about Poetry version mismatch.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Remove unused pytest import and add trailing newline
- Removed unused pytest import (caplog fixture was removed)
- Added missing trailing newline at end of file
Addresses Greptile feedback (minor style issues).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Remove redundant import inside test method
The module litellm.videos.main is already imported at the top of
the file (line 21), so the import inside the test method is redundant.
Addresses Greptile feedback (minor style issue).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Fix converse anthropic usage object according to v1/messages specs
* Add routing based on if reasoning is supported or not
* add fireworks_ai/accounts/fireworks/models/kimi-k2p5 in model map
* Removed stray .md file
* fix(bedrock): clamp thinking.budget_tokens to minimum 1024
Bedrock rejects thinking.budget_tokens values below 1024 with a 400
error. This adds automatic clamping in the LiteLLM transformation
layer so callers (e.g. router with reasoning_effort="low") don't
need to know about the provider-specific minimum.
Fixes#21297
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: improve Langfuse test isolation to prevent flaky failures (#21093)
The test was creating fresh mocks but not fully isolating from setUp state,
causing intermittent CI failures with 'Expected generation to be called once.
Called 0 times.'
Instead of creating fresh mocks, properly reset the existing setUp mocks to
ensure clean state while maintaining proper mock chain configuration.
* feat(s3): add support for virtual-hosted-style URLs (#21094)
Add s3_use_virtual_hosted_style parameter to support AWS S3 virtual-hosted-style URL format (bucket.endpoint/key) alongside the existing path-style format (endpoint/bucket/key).
This enables compatibility with S3-compatible services like MinIO and aligns with AWS S3 official terminology.
* Addressed greptile comments to extract common helpers and return 404
* Allow effort="max" for Claude Opus 4.6 (#21112)
* fix(aiohttp): prevent closing shared ClientSession in AiohttpTransport (#21117)
When a shared ClientSession is passed to LiteLLMAiohttpTransport,
calling aclose() on the transport would close the shared session,
breaking other clients still using it.
Add owns_session parameter (default True for backwards compatibility)
to AiohttpTransport and LiteLLMAiohttpTransport. When a shared session
is provided in http_handler.py, owns_session=False is set to prevent
the transport from closing a session it does not own.
This aligns AiohttpTransport with the ownership pattern already used
in AiohttpHandler (aiohttp_handler.py).
* perf(spend): avoid duplicate daily agent transaction computation (#21187)
* fix: proxy/batches_endpoints/endpoints.py:309:11: PLR0915 Too many statements (54 > 50)
* fix mypy
* Add doc for OpenAI Agents SDK with LiteLLM
* Add doc for OpenAI Agents SDK with LiteLLM
* Update docs/my-website/sidebars.js
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix mypy
* Update tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add blog fffor Managing Anthropic Beta Headers
* Add blog fffor Managing Anthropic Beta Headers
* correct the time
* Fix: Exclude tool params for models without function calling support (#21125) (#21244)
* Fix tool params reported as supported for models without function calling (#21125)
JSON-configured providers (e.g. PublicAI) inherited all OpenAI params
including tools, tool_choice, function_call, and functions — even for
models that don't support function calling. This caused an inconsistency
where get_supported_openai_params included "tools" but
supports_function_calling returned False.
The fix checks supports_function_calling in the dynamic config's
get_supported_openai_params and removes tool-related params when the
model doesn't support it. Follows the same pattern used by OVHCloud
and Fireworks AI providers.
* Style: move verbose_logger to module-level import, remove redundant try/except
Address review feedback from Greptile bot:
- Move verbose_logger import to top-level (matches project convention)
- Remove redundant try/except around supports_function_calling() since it
already handles exceptions internally via _supports_factory()
* fix(index.md): cleanup str
* fix(proxy): handle missing DATABASE_URL in append_query_params (#21239)
* fix: handle missing database url in append_query_params
* Update litellm/proxy/proxy_cli.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(mcp): revert StreamableHTTPSessionManager to stateless mode (#21323)
PR #19809 changed stateless=True to stateless=False to enable progress
notifications for MCP tool calls. This caused the mcp library to enforce
mcp-session-id headers on all non-initialize requests, breaking MCP
Inspector, curl, and any client without automatic session management.
Revert to stateless=True to restore compatibility with all MCP clients.
The progress notification code already handles missing sessions gracefully
(defensive checks + try/except), so no other changes are needed.
Fixes#20242
* UI - Content Filters, help edit/view categories and 1-click add categories + go to next page (#21223)
* feat(ui/): allow viewing content filter categories on guardrail info
* fix(add_guardrail_form.tsx): add validation check to prevent adding empty content filter guardrails
* feat(ui/): improve ux around adding new content filter categories
easy to skip adding a category, so make it a 1-click thing
* Fix OCI Grok output pricing (#21329)
* fix(proxy): fix master key rotation Prisma validation errors
_rotate_master_key() used jsonify_object() which converts Python dicts
to JSON strings. Prisma's Python client rejects strings for Json-typed
fields — it requires prisma.Json() wrappers or native dicts.
This affected three code paths:
- Model table (create_many): litellm_params and model_info converted to
strings, plus created_at/updated_at were None (non-nullable DateTime)
- Config table (update): param_value converted to string
- Credentials table (update): credential_values/credential_info
converted to strings
Fix: replace jsonify_object() with model_dump(exclude_none=True) +
prisma.Json() wrappers for all Json fields. Wrap model delete+insert
in a Prisma transaction for atomicity. Add try/except around MCP
server rotation to prevent non-critical failures from blocking the
entire rotation.
---------
Co-authored-by: Harshit Jain <harshitjain0562@gmail.com>
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Ephrim Stanley <ephrim.stanley@point72.com>
Co-authored-by: Jay Prajapati <79649559+jayy-77@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Julio Quinteros Pro <jquinter@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: mjkam <mjkam@naver.com>
Co-authored-by: Fly <48186978+tuzkiyoung@users.noreply.github.com>
Co-authored-by: Kristoffer Arlind <13228507+KristofferArlind@users.noreply.github.com>
Co-authored-by: Constantine <Runixer@gmail.com>
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: Atharva Jaiswal <92455570+AtharvaJaiswal005@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Fixes#21331 — the Responses API streaming bridge hardcoded index=0 for
all tool call chunks, making parallel tool calls indistinguishable.
Now reads output_index from the Responses API chunk instead.
Resolve conflict in test_converse_transformation.py by keeping both
the structured outputs tests (from this branch) and the
TestBedrockMinThinkingBudgetTokens tests (from main).
When using the Responses API (e.g., Azure gpt-5.1-codex-mini), the response.completed
event was always returning finish_reason='stop', even when the response contained
function_call items in its output. This caused agents like OpenCode to incorrectly
conclude the stream ended without tools to execute, breaking tool/function calling
workflows.
The fix inspects the response.output field in the response.completed event to determine
the correct finish_reason:
- 'tool_calls' when output contains function_call items
- 'stop' otherwise (text-only responses)
Added tests to verify:
- response.completed with function_call output returns finish_reason='tool_calls'
- response.completed with message-only output returns finish_reason='stop'
- response.completed with empty output returns finish_reason='stop' (backward compat)
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* feat(ui/): allow viewing content filter categories on guardrail info
* fix(add_guardrail_form.tsx): add validation check to prevent adding empty content filter guardrails
* feat(ui/): improve ux around adding new content filter categories
easy to skip adding a category, so make it a 1-click thing
PR #19809 changed stateless=True to stateless=False to enable progress
notifications for MCP tool calls. This caused the mcp library to enforce
mcp-session-id headers on all non-initialize requests, breaking MCP
Inspector, curl, and any client without automatic session management.
Revert to stateless=True to restore compatibility with all MCP clients.
The progress notification code already handles missing sessions gracefully
(defensive checks + try/except), so no other changes are needed.
Fixes#20242
* fix: preserve metadata for custom callbacks on codex/responses path (#21204)
- Use metadata or litellm_metadata when calling update_environment_variables
in responses/main.py so metadata is not overwritten by None on the
bridge path (completion -> responses API).
- Add tests for metadata in custom callback for codex models and for
litellm_metadata in aresponses().
Co-authored-by: Cursor <cursoragent@cursor.com>
* Update tests/test_litellm/responses/test_metadata_codex_callback.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Fix tool params reported as supported for models without function calling (#21125)
JSON-configured providers (e.g. PublicAI) inherited all OpenAI params
including tools, tool_choice, function_call, and functions — even for
models that don't support function calling. This caused an inconsistency
where get_supported_openai_params included "tools" but
supports_function_calling returned False.
The fix checks supports_function_calling in the dynamic config's
get_supported_openai_params and removes tool-related params when the
model doesn't support it. Follows the same pattern used by OVHCloud
and Fireworks AI providers.
* Style: move verbose_logger to module-level import, remove redundant try/except
Address review feedback from Greptile bot:
- Move verbose_logger import to top-level (matches project convention)
- Remove redundant try/except around supports_function_calling() since it
already handles exceptions internally via _supports_factory()
