* fix(middleware): replace BaseHTTPMiddleware with pure ASGI middleware
BaseHTTPMiddleware wraps streaming responses with receive_or_disconnect
per chunk, blocking the event loop and causing severe throughput
degradation under concurrent streaming load (53% of CPU in profiling).
Converts PrometheusAuthMiddleware to a pure ASGI middleware using the
__call__(scope, receive, send) protocol.
* fix(streaming): remove expensive debug logging and optimize usage stripping
- Remove print_verbose calls that format chunk/response Pydantic objects,
triggering millions of __repr__ calls (8% of CPU in profiling)
- Guard remaining verbose_logger.debug with isEnabledFor(DEBUG) and use
lazy %s formatting instead of f-strings
- Replace usage stripping round-trip (model_dump + delete + reconstruct)
with a _usage_stripped flag, deferring exclusion to serialization time
* fix(proxy): remove per-chunk debug log and use _usage_stripped flag
- Remove verbose_proxy_logger.debug that formatted every streaming chunk
- Honor _usage_stripped flag from streaming handler to exclude usage
during model_dump_json serialization instead of reconstructing objects
* fix(proxy): remove per-chunk debug log in async_data_generator
Remove verbose_proxy_logger.debug that formatted every streaming chunk,
which triggered expensive Pydantic serialization on the hot path.
* fix indentation and add clarifying comment for usage stripping
* fix: guard calculate_total_usage against None usage in chunks
* fix: store chunk copy to preserve usage for calculate_total_usage
- test_litellm_pre_call_utils.py: wrap test body in try/finally so
litellm.callbacks is always restored even when an assertion fails,
addressing greptile review comment
- test_langfuse_otel.py: resolve trivial merge conflict in comment
("unpatched" vs "unpatch-ed"), keeping correct spelling
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test_reload_model_cost_map_admin_access calls the /reload/model_cost_map
HTTP endpoint with get_model_cost_map mocked to return a single-entry
dict. The endpoint handler does a direct module-level assignment
(litellm.model_cost = new_model_cost_map) which persists after the
patch context manager exits, stripping all models except gpt-3.5-turbo
from the in-memory cost map and causing subsequent tests that rely on
models like gemini-1.5-flash, multimodalembedding@001, and gpt-4o to
fail with "model not mapped" errors or zero-cost spend payloads.
Fix: save litellm.model_cost before the test and restore it (along with
invalidating the case-insensitive lookup cache) in a finally block.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three test isolation issues fixed:
1. test_mcp_debug.py: Replace deprecated asyncio.get_event_loop().run_until_complete()
with asyncio.run() in TestWrapSendWithDebugHeaders. In Python 3.10+,
get_event_loop() raises RuntimeError when no event loop is set in the
current thread, causing test_injects_headers and test_body_messages_unchanged
to fail in isolation.
2. test_mcp_server_manager.py: After _reload_mcp_manager_module() creates a new
global_mcp_server_manager instance, server.py still holds a stale reference
to the old instance. Tests in test_mcp_server.py that populate the new
manager's registry and then call server.py functions (e.g. _get_tools_from_mcp_servers)
get empty results because server.py reads from the old manager. Fix: update
server.py's module-level reference after each reload.
3. test_litellm_pre_call_utils.py: test_add_litellm_metadata_from_request_headers
sets litellm.callbacks without restoring it afterward. Add cleanup to restore
original callbacks after the test to prevent state leaking to subsequent tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Four finally blocks in test_internal_user_endpoints.py and one in
test_ui_sso.py used the pattern:
if original_default_params is not None:
litellm.default_internal_user_params = original_default_params
else:
delattr(litellm, "default_internal_user_params")
Since the attribute is defined in litellm/__init__.py with a default of
None, `getattr(litellm, "default_internal_user_params", None)` returns
None. The else branch then calls delattr(), permanently removing the
attribute from the module for the rest of the process.
Subsequent tests in the same pytest-xdist worker (e.g.
test_add_new_member_* in test_management_helpers_utils.py) then fail
with: AttributeError: module 'litellm' has no attribute
'default_internal_user_params'
Fix: replace all five flawed finally blocks with a simple assignment:
litellm.default_internal_user_params = original_default_params
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The test was using setattr() to set module-level attributes (including
proxy_logging_obj = MagicMock()) on the real litellm.proxy.proxy_server
module, but the finally block only had `pass` — no cleanup.
This left proxy_logging_obj as a MagicMock in subsequent tests running
in the same pytest-xdist worker, causing TypeError when log_db_metrics
decorator called asyncio.create_task(proxy_logging_obj.service_logging_obj
.async_service_success_hook(...)) — a MagicMock is not a coroutine.
Fix: save original attribute values before the test and restore them in
the finally block to ensure test isolation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(schema.prisma): add object permissions for end users
allows controlling if end user can call specific mcp servers
* feat: cleanup for customer_endpoints support of object permission id
* fix: cleanup str
* feat(customers/): enforce end user can only call allowed mcps - if configured
* docs: document customer/end user object permission usage
* feat: address greptile comments
* Add MCP_SECURITY enum to SupportedGuardrailIntegrations
* Add MCP security guardrail initializer
* Add MCPSecurityGuardrail implementation
* Add MCP Security policy template
* Add Type filter to policy templates UI
* Add unit tests for MCP security guardrail
* fix(lint): remove unused Dict import from mcp_security_guardrail
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add French language support for EU AI Act Article 5 guardrail (#21427)
* Add French language support for EU AI Act Article 5 template
- Create eu_ai_act_article5_fr.yaml with comprehensive French keywords
- Includes identifier words: concevoir, créer, développer, noter, classer, etc.
- Includes block words: crédit social, comportement social, émotion des employés, etc.
- Includes always-block keywords for explicit prohibited practices
- Includes exceptions for research, compliance, and legitimate use cases
- Catches circumvention attempts with phrase variations
* Add comprehensive tests for French EU AI Act guardrail
- Test 3 critical scenarios: blocked query, circumvention attempt, safe query
- Test edge cases: case-insensitive, mixed language, research exceptions
- All 7 tests passing
- Validates both blocking and allowing behavior
* Fix content filter to support conditional matching without inherit_from
- Enable conditional matching when identifier_words + additional_block_words are present
- Previously required inherit_from, but EU AI Act templates are self-contained
- Fixes Greptile feedback: conditional matching now works as documented
* Add pure conditional matching test for French guardrail
- Test identifier + block word combinations not in always_block_keywords
- Verifies conditional matching works independently
- Addresses Greptile feedback about test coverage gap
* Fix exception word bypass risk in French template
- Replace short words (film, jeu, juste) with context-specific phrases
- Prevents substring matching bypasses (e.g., enjeu matching jeu)
- Add tests for bypass prevention and legitimate game context
- Addresses Greptile security feedback
* Make conditional match assertion more robust
- Use getattr to safely access exception detail field
- Check if detail is dict before calling .get()
- Addresses Greptile feedback about brittle string assertion
* Add French EU AI Act Article 5 policy template to registry
- Add eu-ai-act-article5-fr template for French language support
- Includes French description and guardrail info
- Matches structure of English template
* Address greptile review feedback (greploop iteration 1)
- Use status_code=400 instead of 403 to match guardrail logging convention
- Use prefix stripping instead of split('/')[-1] for robust server name extraction
* remove French EU AI Act template from policy_templates.json
---------
Co-authored-by: Julio Quinteros Pro <jquinter@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Use importlib.import_module + reload uniformly in both code paths
to ensure fresh module state regardless of whether litellm was
previously in sys.modules. This fixes the inconsistency where the
"not in sys.modules" branch didn't reload the module.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- test_pillar_guardrails.py: Fix fixture to properly update module-level
litellm reference using global keyword and assignment from reload
- test_anthropic_experimental_pass_through_messages_handler.py: Add missing
assert keywords to kwargs comparison statements (lines 36, 60-62)
- test_proxy_server.py: Replace silent pytest.skip with explicit assertion
to catch router initialization regressions
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* allow filtering by user in global usage
* add server root path test to github actions
* Update .github/workflows/test_server_root_path.yml
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* address greptile review feedback (greploop iteration 1)
- Fix HTTPException swallowed by broad except block in get_user_daily_activity
and get_user_daily_activity_aggregated: re-raise HTTPException before the
generic handler so 403 status codes propagate correctly
- Add status_code assertions in non-admin access tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* address greptile review feedback (greploop iteration 2)
- Default user_id to caller's own ID for non-admins instead of 403 when
omitted, preserving backward compatibility for API consumers
- Apply same fix to aggregated endpoint
- Update test to verify defaulting behavior instead of expecting 403
- Add useEffect to sync selectedUserId when auth state settles in
UsagePageView to handle async auth initialization
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fixing syntax
* remove artifacts
* feat: guardrail tracing UI - policy, detection method, match details (#21349)
* feat: add GuardrailTracingDetail TypedDict and tracing fields to StandardLoggingGuardrailInformation
* feat: add policy_template field to Guardrail config TypedDict
* feat: accept GuardrailTracingDetail in base guardrail logging method
* feat: populate tracing fields in content filter guardrail
* test: add tracing fields tests for custom guardrail base class
* test: add tracing fields e2e tests for content filter guardrail
* feat: add guardrail tracing UI - policy badges, match details, timeline
* feat: redesign GuardrailViewer to Guardrails & Policy Compliance layout
Two-column layout with request lifecycle timeline on the left
and compact evaluation detail cards on the right. Header shows
guardrail count, pass/fail status, total overhead, policy info,
and an export button.
* feat: add clickable guardrail link in metrics + show policy names
* feat: add risk_score field to StandardLoggingGuardrailInformation
* feat: compute risk_score in content filter guardrail
* feat: display backend risk_score badge on evaluation cards
* fix: fallback to frontend risk score when backend doesn't provide one
* passing in masster key for api calls
* Fix: Add blog as incident report
* Fix: Add blog as incident report
* remove timeline
* feat(models): add github_copilot/gpt-5.3-codex and github_copilot/claude-opus-4.6-fast (#21316)
Add missing GitHub Copilot model entries for gpt-5.3-codex (GA) and
claude-opus-4.6-fast (Public Preview) to both the root and backup
model pricing JSON files.
* only tests for /ui
* bump: version 1.81.12 → 1.81.13
* Fixing mapped tests
* fixing no_config test
* fixing container tests
* fixing test_basic_openai_responses_api
* Adding bedrock thinking budget tokens to docs
* fixing regen key tests
* fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS
Add store, prompt_cache_key, prompt_cache_retention, safety_identifier, and verbosity
to OPENAI_CHAT_COMPLETION_PARAMS list.
These params were already in DEFAULT_CHAT_COMPLETION_PARAM_VALUES but missing from
the OPENAI_CHAT_COMPLETION_PARAMS list, causing them to be dropped when passed to
OpenAI-compatible providers.
---------
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: Cesar Garcia <128240629+Chesars@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
- Default user_id to caller's own ID for non-admins instead of 403 when
omitted, preserving backward compatibility for API consumers
- Apply same fix to aggregated endpoint
- Update test to verify defaulting behavior instead of expecting 403
- Add useEffect to sync selectedUserId when auth state settles in
UsagePageView to handle async auth initialization
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix HTTPException swallowed by broad except block in get_user_daily_activity
and get_user_daily_activity_aggregated: re-raise HTTPException before the
generic handler so 403 status codes propagate correctly
- Add status_code assertions in non-admin access tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add 6 new EU PII patterns for GDPR compliance
- fr_nir: French Social Security Number (NIR/INSEE) with validation
- eu_iban_enhanced: Enhanced IBAN detection with specific format
- fr_phone: French phone numbers (+33, 0033, 0 formats)
- eu_vat: EU VAT identification numbers (all 27 member states)
- eu_passport_generic: Generic EU passport format
- fr_postal_code: French postal codes with contextual keywords
* Add GDPR Art. 32 EU PII Protection policy template
- Comprehensive GDPR Article 32 compliance policy
- 4 guardrail groups: National IDs, Financial, Contact Info, Business IDs
- Masks French NIR/INSEE, EU IBANs, French phones, EU VAT numbers
- Includes EU passport numbers and email addresses
- Medium complexity template with indigo icon
* Add comprehensive tests for EU PII patterns
- Test French NIR validation (sex digit, month range)
- Test enhanced IBAN detection (French, German)
- Test French phone number formats
- Test EU VAT numbers
- Test generic EU passport format
- Test French postal code pattern
* Add EU pattern loading and category validation tests
- Verify all 6 EU PII patterns are loaded correctly
- Verify patterns are categorized as 'EU PII Patterns'
- Ensure pattern loading consistency
* Add end-to-end tests for GDPR policy template
- 4 tests for PII that should be masked (NIR, IBAN, phone, VAT)
- 4 tests for text that should pass through (invalid patterns, no PII)
- 1 bonus test for multiple PII types in same message
- All tests verify correct masking behavior
* Add region field to policy templates
- Added region field to all 6 templates (EU, AU, Global)
- Updated both main and backup JSON files
- Enables region-based filtering in UI
* Add region filter to policy templates UI
- Added Radio.Group filter for regions (All, AU, EU, Global)
- Efficient filtering with useMemo hooks
- Clean button-based UI matching existing design
- Defaults missing regions to Global
* Apply suggestion from @greptile-apps[bot]
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Address Greptile review: add contextual guards and negative tests
- Added keyword_pattern to eu_vat (VAT, tax number, fiscal code, etc.)
- Added keyword_pattern to eu_passport_generic (passport, travel document, etc.)
- Added 3 negative unit tests for false positive prevention
- Added 2 E2E tests verifying no masking without keyword context
- All patterns now require contextual keywords to prevent false positives
* address greptile review feedback (greploop iteration 1)
- Remove unused HTTPException import from test file
- Add keyword_pattern to eu_vat for contextual VAT matching
- Add allow_word_numbers: false to eu_passport_generic
- Add negative test cases for EU VAT false positives
- All 5 Greptile comments addressed
* Address Greptile feedback: fix patterns and sync backup
- Fix fr_phone pattern: use negative lookbehind (?<!\d) to prevent false matches in longer digit strings
- Add keyword_pattern to eu_passport_generic to reduce false positives on version strings/SKUs
- Sync policy_templates_backup.json with main file (add GDPR template)
- Add keyword_pattern to eu_vat (was auto-added by formatter)
All pattern tests passing
* address greptile review feedback (greploop iteration 2)
- Update test to document that eu_vat raw pattern is intentionally broad
- Test verifies pattern DOES match common words (by design)
- Documents that keyword_pattern guard prevents false positives in production
- Addresses Greptile's false positive risk concern
* address greptile review feedback (greploop iteration 3)
- Fix test_eu_vat_masked: change gap from 2 words to 1 word (VAT number: FR...)
- This ensures keyword matching works within MAX_KEYWORD_VALUE_GAP_WORDS=1 limit
- fr_phone pattern already works correctly (verified with tests)
- test_pattern_requires_keyword_context already updated in iteration 2
Addresses final issues from Greptile 2/5 review
* fix: remove country-specific passport patterns from GDPR template
- Remove passport_france, passport_germany, passport_netherlands from template
- These patterns lack keyword guards and cause false positives
- Only eu_passport_generic remains (has keyword_pattern guard)
- Sync policy_templates_backup.json with main file
- Update test setup to match template
All 11 E2E tests now passing ✅
* Update tests/test_litellm/proxy/guardrails/guardrail_hooks/content_filter/test_gdpr_policy_e2e.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/content_filter.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: SSO PKCE support fails in multi-pod Kubernetes deployments
* fix: virutal key grace period from env/UI
* fix: refactor, race condition handle, fstring sql injection
* fix: add async call to avoid server pauses
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: add await in tests
* add modify test to perform async run
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix grace period with better error handling on frontend and as per best practices
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: as per request changes
* Update litellm/proxy/utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Fix errors when callbacks are invoked for file delete operations:
* Fix errors when callbacks are invoked for file operations
* Fix: pass deployment credentials to afile_retrieve in managed_files post-call hook
* Fix: bypass managed files access check in batch polling by calling afile_content directly
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: afile_retrieve returns unified ID for batch output files
* fix: batch retrieve returns unified input_file_id
* fix(chatgpt): drop unsupported responses params for Codex
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(chatgpt): ensure Codex request filters unsupported params
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix deleted managed files returning 403 instead of 404
* Add comments
* Update litellm/proxy/utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: thread deployment model_info through batch cost calculation
batch_cost_calculator only checked the global cost map, ignoring
deployment-level custom pricing (input_cost_per_token_batches etc.).
Add optional model_info param through the batch cost chain and pass
it from CheckBatchCost.
* fix(deps): add pytest-postgresql for db schema migration tests
The test_db_schema_migration.py test requires pytest-postgresql but it was
missing from dependencies, causing import errors:
ModuleNotFoundError: No module named 'pytest_postgresql'
Added pytest-postgresql ^6.0.0 to dev dependencies to fix test collection
errors in proxy_unit_tests.
This is a pre-existing issue, not related to PR #21277.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): replace caplog with custom handler for parallel execution
The cost calculation log level tests were failing when run with pytest-xdist
parallel execution because caplog doesn't work reliably across worker processes.
This causes "ValueError: I/O operation on closed file" errors.
Solution: Replace caplog fixture with a custom LogRecordHandler that directly
attaches to the logger. This approach works correctly in parallel execution
because each worker process has its own handler instance.
Fixes test failures in PR #21277 when running with --dist=loadscope.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): correct async mock for video generation logging test
The test was failing with AuthenticationError because the mock wasn't
intercepting the actual HTTP handler calls. This caused real API calls
with no API key, resulting in 401 errors.
Root cause: The test was patching the wrong target using string path
'litellm.videos.main.base_llm_http_handler' instead of using patch.object
on the actual handler instance. Additionally, it was mocking the sync
method instead of async_video_generation_handler.
Solution: Use patch.object with side_effect pattern on the correct
async handler method, following the same pattern used in
test_video_generation_async().
Fixes test failure in PR #21277 when running with --dist=loadscope.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): add cleanup fixture and no_parallel mark for MCP tests
Two MCP server tests were failing when run with pytest-xdist parallel
execution (--dist=loadscope):
- test_mcp_routing_with_conflicting_alias_and_group_name
- test_oauth2_headers_passed_to_mcp_client
Both tests showed assertion failures where mocks weren't being called
(0 times instead of expected 1 time).
Root cause: These tests rely on global_mcp_server_manager singleton
state and complex async mocking that doesn't work reliably with
parallel execution. Each worker process can have different state
and patches may not apply correctly.
Solution:
1. Added autouse fixture to clean up global_mcp_server_manager registry
before and after each test for better isolation
2. Added @pytest.mark.no_parallel to these specific tests to ensure
they run sequentially, avoiding parallel execution issues
This approach maintains test reliability while allowing other tests
in the file to still benefit from parallelization.
Fixes test failures exposed by PR #21277.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Regenerate poetry.lock with Poetry 2.3.2
Updated lock file to use Poetry 2.3.2 (matching main branch standard).
This addresses Greptile feedback about Poetry version mismatch.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Remove unused pytest import and add trailing newline
- Removed unused pytest import (caplog fixture was removed)
- Added missing trailing newline at end of file
Addresses Greptile feedback (minor style issues).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Remove redundant import inside test method
The module litellm.videos.main is already imported at the top of
the file (line 21), so the import inside the test method is redundant.
Addresses Greptile feedback (minor style issue).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Fix converse anthropic usage object according to v1/messages specs
* Add routing based on if reasoning is supported or not
* add fireworks_ai/accounts/fireworks/models/kimi-k2p5 in model map
* Removed stray .md file
* fix(bedrock): clamp thinking.budget_tokens to minimum 1024
Bedrock rejects thinking.budget_tokens values below 1024 with a 400
error. This adds automatic clamping in the LiteLLM transformation
layer so callers (e.g. router with reasoning_effort="low") don't
need to know about the provider-specific minimum.
Fixes#21297
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: improve Langfuse test isolation to prevent flaky failures (#21093)
The test was creating fresh mocks but not fully isolating from setUp state,
causing intermittent CI failures with 'Expected generation to be called once.
Called 0 times.'
Instead of creating fresh mocks, properly reset the existing setUp mocks to
ensure clean state while maintaining proper mock chain configuration.
* feat(s3): add support for virtual-hosted-style URLs (#21094)
Add s3_use_virtual_hosted_style parameter to support AWS S3 virtual-hosted-style URL format (bucket.endpoint/key) alongside the existing path-style format (endpoint/bucket/key).
This enables compatibility with S3-compatible services like MinIO and aligns with AWS S3 official terminology.
* Addressed greptile comments to extract common helpers and return 404
* Allow effort="max" for Claude Opus 4.6 (#21112)
* fix(aiohttp): prevent closing shared ClientSession in AiohttpTransport (#21117)
When a shared ClientSession is passed to LiteLLMAiohttpTransport,
calling aclose() on the transport would close the shared session,
breaking other clients still using it.
Add owns_session parameter (default True for backwards compatibility)
to AiohttpTransport and LiteLLMAiohttpTransport. When a shared session
is provided in http_handler.py, owns_session=False is set to prevent
the transport from closing a session it does not own.
This aligns AiohttpTransport with the ownership pattern already used
in AiohttpHandler (aiohttp_handler.py).
* perf(spend): avoid duplicate daily agent transaction computation (#21187)
* fix: proxy/batches_endpoints/endpoints.py:309:11: PLR0915 Too many statements (54 > 50)
* fix mypy
* Add doc for OpenAI Agents SDK with LiteLLM
* Add doc for OpenAI Agents SDK with LiteLLM
* Update docs/my-website/sidebars.js
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix mypy
* Update tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add blog fffor Managing Anthropic Beta Headers
* Add blog fffor Managing Anthropic Beta Headers
* correct the time
* Fix: Exclude tool params for models without function calling support (#21125) (#21244)
* Fix tool params reported as supported for models without function calling (#21125)
JSON-configured providers (e.g. PublicAI) inherited all OpenAI params
including tools, tool_choice, function_call, and functions — even for
models that don't support function calling. This caused an inconsistency
where get_supported_openai_params included "tools" but
supports_function_calling returned False.
The fix checks supports_function_calling in the dynamic config's
get_supported_openai_params and removes tool-related params when the
model doesn't support it. Follows the same pattern used by OVHCloud
and Fireworks AI providers.
* Style: move verbose_logger to module-level import, remove redundant try/except
Address review feedback from Greptile bot:
- Move verbose_logger import to top-level (matches project convention)
- Remove redundant try/except around supports_function_calling() since it
already handles exceptions internally via _supports_factory()
* fix(index.md): cleanup str
* fix(proxy): handle missing DATABASE_URL in append_query_params (#21239)
* fix: handle missing database url in append_query_params
* Update litellm/proxy/proxy_cli.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(mcp): revert StreamableHTTPSessionManager to stateless mode (#21323)
PR #19809 changed stateless=True to stateless=False to enable progress
notifications for MCP tool calls. This caused the mcp library to enforce
mcp-session-id headers on all non-initialize requests, breaking MCP
Inspector, curl, and any client without automatic session management.
Revert to stateless=True to restore compatibility with all MCP clients.
The progress notification code already handles missing sessions gracefully
(defensive checks + try/except), so no other changes are needed.
Fixes#20242
* UI - Content Filters, help edit/view categories and 1-click add categories + go to next page (#21223)
* feat(ui/): allow viewing content filter categories on guardrail info
* fix(add_guardrail_form.tsx): add validation check to prevent adding empty content filter guardrails
* feat(ui/): improve ux around adding new content filter categories
easy to skip adding a category, so make it a 1-click thing
* Fix OCI Grok output pricing (#21329)
* fix(proxy): fix master key rotation Prisma validation errors
_rotate_master_key() used jsonify_object() which converts Python dicts
to JSON strings. Prisma's Python client rejects strings for Json-typed
fields — it requires prisma.Json() wrappers or native dicts.
This affected three code paths:
- Model table (create_many): litellm_params and model_info converted to
strings, plus created_at/updated_at were None (non-nullable DateTime)
- Config table (update): param_value converted to string
- Credentials table (update): credential_values/credential_info
converted to strings
Fix: replace jsonify_object() with model_dump(exclude_none=True) +
prisma.Json() wrappers for all Json fields. Wrap model delete+insert
in a Prisma transaction for atomicity. Add try/except around MCP
server rotation to prevent non-critical failures from blocking the
entire rotation.
---------
Co-authored-by: Harshit Jain <harshitjain0562@gmail.com>
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Ephrim Stanley <ephrim.stanley@point72.com>
Co-authored-by: Jay Prajapati <79649559+jayy-77@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Julio Quinteros Pro <jquinter@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: mjkam <mjkam@naver.com>
Co-authored-by: Fly <48186978+tuzkiyoung@users.noreply.github.com>
Co-authored-by: Kristoffer Arlind <13228507+KristofferArlind@users.noreply.github.com>
Co-authored-by: Constantine <Runixer@gmail.com>
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: Atharva Jaiswal <92455570+AtharvaJaiswal005@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
PR #19809 changed stateless=True to stateless=False to enable progress
notifications for MCP tool calls. This caused the mcp library to enforce
mcp-session-id headers on all non-initialize requests, breaking MCP
Inspector, curl, and any client without automatic session management.
Revert to stateless=True to restore compatibility with all MCP clients.
The progress notification code already handles missing sessions gracefully
(defensive checks + try/except), so no other changes are needed.
Fixes#20242
Two MCP server tests were failing when run with pytest-xdist parallel
execution (--dist=loadscope):
- test_mcp_routing_with_conflicting_alias_and_group_name
- test_oauth2_headers_passed_to_mcp_client
Both tests showed assertion failures where mocks weren't being called
(0 times instead of expected 1 time).
Root cause: These tests rely on global_mcp_server_manager singleton
state and complex async mocking that doesn't work reliably with
parallel execution. Each worker process can have different state
and patches may not apply correctly.
Solution:
1. Added autouse fixture to clean up global_mcp_server_manager registry
before and after each test for better isolation
2. Added @pytest.mark.no_parallel to these specific tests to ensure
they run sequentially, avoiding parallel execution issues
This approach maintains test reliability while allowing other tests
in the file to still benefit from parallelization.
Fixes test failures exposed by PR #21277.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
