* allow filtering by user in global usage
* add server root path test to github actions
* Update .github/workflows/test_server_root_path.yml
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* address greptile review feedback (greploop iteration 1)
- Fix HTTPException swallowed by broad except block in get_user_daily_activity
and get_user_daily_activity_aggregated: re-raise HTTPException before the
generic handler so 403 status codes propagate correctly
- Add status_code assertions in non-admin access tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* address greptile review feedback (greploop iteration 2)
- Default user_id to caller's own ID for non-admins instead of 403 when
omitted, preserving backward compatibility for API consumers
- Apply same fix to aggregated endpoint
- Update test to verify defaulting behavior instead of expecting 403
- Add useEffect to sync selectedUserId when auth state settles in
UsagePageView to handle async auth initialization
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fixing syntax
* remove artifacts
* feat: guardrail tracing UI - policy, detection method, match details (#21349)
* feat: add GuardrailTracingDetail TypedDict and tracing fields to StandardLoggingGuardrailInformation
* feat: add policy_template field to Guardrail config TypedDict
* feat: accept GuardrailTracingDetail in base guardrail logging method
* feat: populate tracing fields in content filter guardrail
* test: add tracing fields tests for custom guardrail base class
* test: add tracing fields e2e tests for content filter guardrail
* feat: add guardrail tracing UI - policy badges, match details, timeline
* feat: redesign GuardrailViewer to Guardrails & Policy Compliance layout
Two-column layout with request lifecycle timeline on the left
and compact evaluation detail cards on the right. Header shows
guardrail count, pass/fail status, total overhead, policy info,
and an export button.
* feat: add clickable guardrail link in metrics + show policy names
* feat: add risk_score field to StandardLoggingGuardrailInformation
* feat: compute risk_score in content filter guardrail
* feat: display backend risk_score badge on evaluation cards
* fix: fallback to frontend risk score when backend doesn't provide one
* passing in masster key for api calls
* Fix: Add blog as incident report
* Fix: Add blog as incident report
* remove timeline
* feat(models): add github_copilot/gpt-5.3-codex and github_copilot/claude-opus-4.6-fast (#21316)
Add missing GitHub Copilot model entries for gpt-5.3-codex (GA) and
claude-opus-4.6-fast (Public Preview) to both the root and backup
model pricing JSON files.
* only tests for /ui
* bump: version 1.81.12 → 1.81.13
* Fixing mapped tests
* fixing no_config test
* fixing container tests
* fixing test_basic_openai_responses_api
* Adding bedrock thinking budget tokens to docs
* fixing regen key tests
* fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS
Add store, prompt_cache_key, prompt_cache_retention, safety_identifier, and verbosity
to OPENAI_CHAT_COMPLETION_PARAMS list.
These params were already in DEFAULT_CHAT_COMPLETION_PARAM_VALUES but missing from
the OPENAI_CHAT_COMPLETION_PARAMS list, causing them to be dropped when passed to
OpenAI-compatible providers.
---------
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: Cesar Garcia <128240629+Chesars@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
PR #21039 fixed OAuth token handling at the LLM layer (Authorization: Bearer
instead of x-api-key), but the proxy layer still strips the Authorization
header in clean_headers() before it reaches the Anthropic code. This breaks
OAuth for proxy users (e.g., Claude Code Max through LiteLLM proxy).
Changes:
- Add is_anthropic_oauth_key() helper to detect OAuth tokens (sk-ant-oat*)
- Preserve OAuth Authorization headers in clean_headers() instead of stripping
- Forward OAuth Authorization via ProviderSpecificHeader in
add_provider_specific_headers_to_request() so tokens only reach
Anthropic-compatible providers (anthropic, bedrock, vertex_ai)
Fixes#19618
Co-authored-by: Adam Reed <iamadamreed@users.noreply.github.com>
* fix: SSO PKCE support fails in multi-pod Kubernetes deployments
* fix: virutal key grace period from env/UI
* fix: refactor, race condition handle, fstring sql injection
* fix: add async call to avoid server pauses
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: add await in tests
* add modify test to perform async run
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix grace period with better error handling on frontend and as per best practices
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: as per request changes
* Update litellm/proxy/utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Fix errors when callbacks are invoked for file delete operations:
* Fix errors when callbacks are invoked for file operations
* Fix: pass deployment credentials to afile_retrieve in managed_files post-call hook
* Fix: bypass managed files access check in batch polling by calling afile_content directly
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: afile_retrieve returns unified ID for batch output files
* fix: batch retrieve returns unified input_file_id
* fix(chatgpt): drop unsupported responses params for Codex
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(chatgpt): ensure Codex request filters unsupported params
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix deleted managed files returning 403 instead of 404
* Add comments
* Update litellm/proxy/utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: thread deployment model_info through batch cost calculation
batch_cost_calculator only checked the global cost map, ignoring
deployment-level custom pricing (input_cost_per_token_batches etc.).
Add optional model_info param through the batch cost chain and pass
it from CheckBatchCost.
* fix(deps): add pytest-postgresql for db schema migration tests
The test_db_schema_migration.py test requires pytest-postgresql but it was
missing from dependencies, causing import errors:
ModuleNotFoundError: No module named 'pytest_postgresql'
Added pytest-postgresql ^6.0.0 to dev dependencies to fix test collection
errors in proxy_unit_tests.
This is a pre-existing issue, not related to PR #21277.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): replace caplog with custom handler for parallel execution
The cost calculation log level tests were failing when run with pytest-xdist
parallel execution because caplog doesn't work reliably across worker processes.
This causes "ValueError: I/O operation on closed file" errors.
Solution: Replace caplog fixture with a custom LogRecordHandler that directly
attaches to the logger. This approach works correctly in parallel execution
because each worker process has its own handler instance.
Fixes test failures in PR #21277 when running with --dist=loadscope.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): correct async mock for video generation logging test
The test was failing with AuthenticationError because the mock wasn't
intercepting the actual HTTP handler calls. This caused real API calls
with no API key, resulting in 401 errors.
Root cause: The test was patching the wrong target using string path
'litellm.videos.main.base_llm_http_handler' instead of using patch.object
on the actual handler instance. Additionally, it was mocking the sync
method instead of async_video_generation_handler.
Solution: Use patch.object with side_effect pattern on the correct
async handler method, following the same pattern used in
test_video_generation_async().
Fixes test failure in PR #21277 when running with --dist=loadscope.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): add cleanup fixture and no_parallel mark for MCP tests
Two MCP server tests were failing when run with pytest-xdist parallel
execution (--dist=loadscope):
- test_mcp_routing_with_conflicting_alias_and_group_name
- test_oauth2_headers_passed_to_mcp_client
Both tests showed assertion failures where mocks weren't being called
(0 times instead of expected 1 time).
Root cause: These tests rely on global_mcp_server_manager singleton
state and complex async mocking that doesn't work reliably with
parallel execution. Each worker process can have different state
and patches may not apply correctly.
Solution:
1. Added autouse fixture to clean up global_mcp_server_manager registry
before and after each test for better isolation
2. Added @pytest.mark.no_parallel to these specific tests to ensure
they run sequentially, avoiding parallel execution issues
This approach maintains test reliability while allowing other tests
in the file to still benefit from parallelization.
Fixes test failures exposed by PR #21277.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Regenerate poetry.lock with Poetry 2.3.2
Updated lock file to use Poetry 2.3.2 (matching main branch standard).
This addresses Greptile feedback about Poetry version mismatch.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Remove unused pytest import and add trailing newline
- Removed unused pytest import (caplog fixture was removed)
- Added missing trailing newline at end of file
Addresses Greptile feedback (minor style issues).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Remove redundant import inside test method
The module litellm.videos.main is already imported at the top of
the file (line 21), so the import inside the test method is redundant.
Addresses Greptile feedback (minor style issue).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Fix converse anthropic usage object according to v1/messages specs
* Add routing based on if reasoning is supported or not
* add fireworks_ai/accounts/fireworks/models/kimi-k2p5 in model map
* Removed stray .md file
* fix(bedrock): clamp thinking.budget_tokens to minimum 1024
Bedrock rejects thinking.budget_tokens values below 1024 with a 400
error. This adds automatic clamping in the LiteLLM transformation
layer so callers (e.g. router with reasoning_effort="low") don't
need to know about the provider-specific minimum.
Fixes#21297
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: improve Langfuse test isolation to prevent flaky failures (#21093)
The test was creating fresh mocks but not fully isolating from setUp state,
causing intermittent CI failures with 'Expected generation to be called once.
Called 0 times.'
Instead of creating fresh mocks, properly reset the existing setUp mocks to
ensure clean state while maintaining proper mock chain configuration.
* feat(s3): add support for virtual-hosted-style URLs (#21094)
Add s3_use_virtual_hosted_style parameter to support AWS S3 virtual-hosted-style URL format (bucket.endpoint/key) alongside the existing path-style format (endpoint/bucket/key).
This enables compatibility with S3-compatible services like MinIO and aligns with AWS S3 official terminology.
* Addressed greptile comments to extract common helpers and return 404
* Allow effort="max" for Claude Opus 4.6 (#21112)
* fix(aiohttp): prevent closing shared ClientSession in AiohttpTransport (#21117)
When a shared ClientSession is passed to LiteLLMAiohttpTransport,
calling aclose() on the transport would close the shared session,
breaking other clients still using it.
Add owns_session parameter (default True for backwards compatibility)
to AiohttpTransport and LiteLLMAiohttpTransport. When a shared session
is provided in http_handler.py, owns_session=False is set to prevent
the transport from closing a session it does not own.
This aligns AiohttpTransport with the ownership pattern already used
in AiohttpHandler (aiohttp_handler.py).
* perf(spend): avoid duplicate daily agent transaction computation (#21187)
* fix: proxy/batches_endpoints/endpoints.py:309:11: PLR0915 Too many statements (54 > 50)
* fix mypy
* Add doc for OpenAI Agents SDK with LiteLLM
* Add doc for OpenAI Agents SDK with LiteLLM
* Update docs/my-website/sidebars.js
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix mypy
* Update tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add blog fffor Managing Anthropic Beta Headers
* Add blog fffor Managing Anthropic Beta Headers
* correct the time
* Fix: Exclude tool params for models without function calling support (#21125) (#21244)
* Fix tool params reported as supported for models without function calling (#21125)
JSON-configured providers (e.g. PublicAI) inherited all OpenAI params
including tools, tool_choice, function_call, and functions — even for
models that don't support function calling. This caused an inconsistency
where get_supported_openai_params included "tools" but
supports_function_calling returned False.
The fix checks supports_function_calling in the dynamic config's
get_supported_openai_params and removes tool-related params when the
model doesn't support it. Follows the same pattern used by OVHCloud
and Fireworks AI providers.
* Style: move verbose_logger to module-level import, remove redundant try/except
Address review feedback from Greptile bot:
- Move verbose_logger import to top-level (matches project convention)
- Remove redundant try/except around supports_function_calling() since it
already handles exceptions internally via _supports_factory()
* fix(index.md): cleanup str
* fix(proxy): handle missing DATABASE_URL in append_query_params (#21239)
* fix: handle missing database url in append_query_params
* Update litellm/proxy/proxy_cli.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(mcp): revert StreamableHTTPSessionManager to stateless mode (#21323)
PR #19809 changed stateless=True to stateless=False to enable progress
notifications for MCP tool calls. This caused the mcp library to enforce
mcp-session-id headers on all non-initialize requests, breaking MCP
Inspector, curl, and any client without automatic session management.
Revert to stateless=True to restore compatibility with all MCP clients.
The progress notification code already handles missing sessions gracefully
(defensive checks + try/except), so no other changes are needed.
Fixes#20242
* UI - Content Filters, help edit/view categories and 1-click add categories + go to next page (#21223)
* feat(ui/): allow viewing content filter categories on guardrail info
* fix(add_guardrail_form.tsx): add validation check to prevent adding empty content filter guardrails
* feat(ui/): improve ux around adding new content filter categories
easy to skip adding a category, so make it a 1-click thing
* Fix OCI Grok output pricing (#21329)
* fix(proxy): fix master key rotation Prisma validation errors
_rotate_master_key() used jsonify_object() which converts Python dicts
to JSON strings. Prisma's Python client rejects strings for Json-typed
fields — it requires prisma.Json() wrappers or native dicts.
This affected three code paths:
- Model table (create_many): litellm_params and model_info converted to
strings, plus created_at/updated_at were None (non-nullable DateTime)
- Config table (update): param_value converted to string
- Credentials table (update): credential_values/credential_info
converted to strings
Fix: replace jsonify_object() with model_dump(exclude_none=True) +
prisma.Json() wrappers for all Json fields. Wrap model delete+insert
in a Prisma transaction for atomicity. Add try/except around MCP
server rotation to prevent non-critical failures from blocking the
entire rotation.
---------
Co-authored-by: Harshit Jain <harshitjain0562@gmail.com>
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Ephrim Stanley <ephrim.stanley@point72.com>
Co-authored-by: Jay Prajapati <79649559+jayy-77@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Julio Quinteros Pro <jquinter@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: mjkam <mjkam@naver.com>
Co-authored-by: Fly <48186978+tuzkiyoung@users.noreply.github.com>
Co-authored-by: Kristoffer Arlind <13228507+KristofferArlind@users.noreply.github.com>
Co-authored-by: Constantine <Runixer@gmail.com>
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: Atharva Jaiswal <92455570+AtharvaJaiswal005@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Resolve conflict in test_converse_transformation.py by keeping both
the structured outputs tests (from this branch) and the
TestBedrockMinThinkingBudgetTokens tests (from main).
* Fix tool params reported as supported for models without function calling (#21125)
JSON-configured providers (e.g. PublicAI) inherited all OpenAI params
including tools, tool_choice, function_call, and functions — even for
models that don't support function calling. This caused an inconsistency
where get_supported_openai_params included "tools" but
supports_function_calling returned False.
The fix checks supports_function_calling in the dynamic config's
get_supported_openai_params and removes tool-related params when the
model doesn't support it. Follows the same pattern used by OVHCloud
and Fireworks AI providers.
* Style: move verbose_logger to module-level import, remove redundant try/except
Address review feedback from Greptile bot:
- Move verbose_logger import to top-level (matches project convention)
- Remove redundant try/except around supports_function_calling() since it
already handles exceptions internally via _supports_factory()
When a shared ClientSession is passed to LiteLLMAiohttpTransport,
calling aclose() on the transport would close the shared session,
breaking other clients still using it.
Add owns_session parameter (default True for backwards compatibility)
to AiohttpTransport and LiteLLMAiohttpTransport. When a shared session
is provided in http_handler.py, owns_session=False is set to prevent
the transport from closing a session it does not own.
This aligns AiohttpTransport with the ownership pattern already used
in AiohttpHandler (aiohttp_handler.py).
Bedrock rejects thinking.budget_tokens values below 1024 with a 400
error. This adds automatic clamping in the LiteLLM transformation
layer so callers (e.g. router with reasoning_effort="low") don't
need to know about the provider-specific minimum.
Fixes#21297
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test_vertex_ai_gpt_oss_simple_request and test_vertex_ai_gpt_oss_reasoning_effort
tests were failing in CI with 401 authentication errors. This was because the
vertexai module import was triggering authentication attempts even though the
_ensure_access_token method was mocked.
Added patch.dict('sys.modules', ...) to mock the vertexai module entirely,
preventing it from trying to authenticate when imported. This ensures tests
are fully isolated and don't attempt real API calls regardless of environment
variables or test execution order.
This follows the same pattern used in other Vertex AI tests and works in
combination with the autouse fixture that clears environment variables.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The test_watsonx_gpt_oss_prompt_transformation was using return_value to mock
an async method (AsyncHTTPHandler.post), which doesn't work correctly with
async/await. This could cause intermittent failures in CI due to test ordering.
Changed to use side_effect with an async function (mock_post_func) to properly
mock the async post method, following the same pattern used in other async
tests like test_vertex_ai_gpt_oss_reasoning_effort.
This ensures the mock is always called correctly regardless of test execution
order or parallel test execution.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add autouse pytest fixture to clear Google/Vertex AI environment
variables before each test, preventing authentication errors in CI.
Previous tests may set GOOGLE_APPLICATION_CREDENTIALS or other Vertex
environment variables and not clean them up, causing this test to
attempt real Google authentication instead of using mocks.
This fix:
- Adds clean_vertex_env fixture with autouse=True
- Saves and clears Google/Vertex env vars before each test
- Restores them after each test
- Prevents "AuthenticationError: Request had invalid authentication
credentials" (401) in CI when run with other tests
Same fix pattern as PR #21268 (rerank) and PR #21272 (GPT-OSS).
Related: test was failing on PR #21217, but NOT caused by PR #21217
(which only modifies test_anthropic_structured_output.py). This is
another test isolation issue.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add autouse pytest fixture to clear Google/Vertex AI environment
variables before each test, preventing authentication errors in CI.
Previous tests may set GOOGLE_APPLICATION_CREDENTIALS or other Vertex
environment variables and not clean them up, causing this test to
attempt real Google authentication instead of using mocks.
This fix:
- Adds clean_vertex_env fixture with autouse=True
- Saves and clears Google/Vertex env vars before each test
- Restores them after each test
- Prevents "AuthenticationError: Request had invalid authentication
credentials" in CI when run with other tests
Test makes real API calls in CI without this fix, gets 401 error.
Locally fails with "No module named 'vertexai'" (expected).
Related: test was failing on PR #21217, but NOT caused by PR #21217
(which only modifies test_anthropic_structured_output.py). This is
another test isolation issue.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
PR #20813 changed the Anthropic schema filter to remove string and numeric constraints
(minLength/maxLength, minimum/maximum) per Anthropic API requirements, but forgot to
update the corresponding test.
The new behavior (per Anthropic SDK):
1. Remove unsupported constraints from schema (Anthropic API doesn't support them)
2. Add constraint info to description field (e.g., "Note: minimum length: 1")
**Changes:**
- Updated test to expect constraints REMOVED from schema
- Added assertions to verify constraints are added to description
- Updated docstring to explain the new behavior
**Testing:**
- ✅ test_other_constraints_preserved now passes
- ✅ All 4 tests in test_anthropic_structured_output.py pass
**Related:**
- Fixes test broken by PR #20813
- Aligns with Anthropic API requirements documented in commit 84934a7258
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Update test expectations to match the current code behavior where
reasoning_effort is transformed from a string to a dict with
'effort' and 'summary' fields.
The transformation happens in:
litellm/llms/anthropic/experimental_pass_through/adapters/handler.py:72-74
When reasoning_effort is a string like "minimal", it's converted to:
{"effort": "minimal", "summary": "detailed"}
The test was expecting just the string "minimal", causing it to fail.
Test now passes ✅
Related: test was failing on PR #21217, but NOT caused by PR #21217
(which only modifies test_anthropic_structured_output.py). This is a
pre-existing broken test that also fails on main branch.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add setup_method and teardown_method to clean up Google/Vertex AI
environment variables that may be left by previous tests.
Previous tests may set GOOGLE_APPLICATION_CREDENTIALS or other Vertex
environment variables and not clean them up, causing this test to
attempt real Google authentication instead of using mocks.
This fix:
- Saves and clears Google/Vertex env vars in setup_method
- Restores them in teardown_method
- Prevents "DefaultCredentialsError" in CI when run with other tests
Test passes in isolation but fails in CI due to test ordering. This
is another test isolation issue, NOT related to PR #21217.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The test_end_to_end_rerank_flow mock for _ensure_access_token was not
being applied because conftest reloads litellm, causing the
VertexAIRerankConfig class to be a different object than what's patched.
Fix: Reload the transformation module in setup_method and re-import the
class to ensure the patch targets the same class object used by tests.
The tests were making real API calls instead of using mocks because
conftest.py reloads litellm at module scope, causing the HTTPHandler
class reference in the HuggingFace embedding handler to become stale.
The patches were applied to the new class, but the handler used the old one.
Fix: Add a reload_huggingface_modules fixture that reloads the relevant
modules BEFORE the mock fixtures apply their patches. This ensures all
references point to the same class object.
Fix several tests that fail in CI due to parallel test execution and
module reloading in conftest.py.
1. test_empty_assistant_message_handling:
- Use patch.object on factory_module.litellm instead of direct assignment
- Ensures the correct litellm reference is modified after conftest reloads
2. test_embedding_header_forwarding_with_model_group:
- Use patch.object on pre_call_utils_module.litellm instead of direct assignment
- Same fix for module reloading issue
3. test_embedding_input_array_of_tokens:
- Move mock inside test function (after fixture initializes router)
- Add skip condition if llm_router is None
- Fixes "AttributeError: None does not have 'aembedding'" in parallel execution
Root cause: conftest.py reloads litellm at module scope, which can cause:
- Different litellm references between test code and library code
- Global state (like llm_router) being None at decorator execution time
- isinstance checks failing due to class identity mismatches
1. test_bedrock_converse_budget_tokens_preserved:
- Fixed mocking at the correct level (litellm.acompletion instead of client.post)
- The previous mock didn't work because the code runs through run_in_executor
and the passed client parameter was not being used
2. test_error_class_returns_volcengine_error:
- Changed isinstance check to class name comparison
- This avoids issues when module reloading (in conftest.py) causes class
identity mismatches during parallel test execution
This commit addresses two issues:
1. **Merge conflict resolution**: Resolved merge conflict in litellm/integrations/opentelemetry.py
that was preventing imports from working. The conflict was in the OpenTelemetry SDK
LogRecord import section.
2. **Test flakiness fix**: Fixed intermittent failures in test_bedrock_converse_budget_tokens_preserved
by properly configuring mock objects to avoid unawaited coroutine warnings.
The test was failing in CI with "Expected 'post' to have been called once. Called 0 times."
The root cause was improper mock setup where AsyncMock was creating async child methods
(raise_for_status, json) that returned unawaited coroutines, causing unreliable behavior
across different Python versions and test environments.
**Changes:**
- Set raise_for_status() and json() as explicit MagicMock instances on the response
- Use AsyncMock explicitly for the post() method via patch.object's 'new' parameter
- This ensures response methods are synchronous while the HTTP call remains async
**Testing:**
- Test now passes consistently across 5 consecutive runs
- RuntimeWarnings about unawaited coroutines eliminated (18 warnings → 16 warnings)
- Request JSON verification shows budget_tokens correctly preserved
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add `definitions` handling alongside `$defs` in schema normalization
(older JSON Schema drafts use `definitions` instead of `$defs`)
- Fall back to tool-call approach when `response_format: {type: json_object}`
has no explicit schema, since the native API requires one
- Add tests for both cases
* fix(model_cost): add missing supports_system_messages and supports_tool_choice to bedrock/moonshotai.kimi-k2.5
* fix(streaming): ensure role=assistant is set on first streaming chunk via strip_role_from_delta
* fix(vertex_ai): ensure role=assistant on first streaming chunk for Llama models
Add VertexAILlama3StreamingHandler that injects role='assistant' into the
first streaming chunk delta when the Vertex AI Llama API omits it.
* docs: add reference to example_openai_endpoint repo for self-hosting fake OpenAI proxy (#21006)
- Updated benchmarks.md with a section on setting up fake OpenAI endpoints
- Updated load_test.md to mention the self-hosted option
- Updated load_test_advanced.md with a tip box about the example repo
Reference: https://github.com/BerriAI/example_openai_endpoint
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* MCP fixes
* fix(oldteams.tsx): show policies when creating
* fix(proxy/_types.py): ensure mcp rest endpoints can be called by virtual key
ensures UI works with virtual key testing mcp endpoints
* refactor: migrate get object permissions table logic to happen in user api key auth - allows functions to trust user api key object they receive has what they need
* fix(rest_endpoints.py): filter for allowed tools based on what key has access to
* fix(mcp_server_manager.py): ensure only allowed MCP's are returned to the user, via rest endpoints
* Guardrails - add toxic/abusive content filter guardrails
* fix(streaming): preserve usage data from post-finish_reason chunks in OpenAI-compatible streaming
Fixes#16112
OpenRouter and other OpenAI-compatible providers send a usage chunk after
the finish_reason='stop' chunk when stream_options.include_usage is True.
The OpenAIChatCompletionStreamingHandler.chunk_parser() was not passing
the usage field to ModelResponseStream, causing real token counts from the
provider to be lost and falling back to inaccurate estimates.
* fix: resolve merge conflict in test file
- Fix typo in test method name (extra space)
- Move test_prompt_cache_key_in_optional_params to its own class
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* fix: allow Management keys to access user/daily/activity and team/daily/activity
* feat(vertex): surface trafficType via generic provider_specific_fields in Responses API
Extract Vertex AI's trafficType from usageMetadata in both streaming and
non-streaming paths, storing it in _hidden_params["provider_specific_fields"].
The Responses API transformation layer generically passes any
_hidden_params["provider_specific_fields"] dict to the ResponsesAPIResponse,
avoiding provider-specific logic in the bridge.
Also fix stream_chunk_builder to propagate _hidden_params from the last
streaming chunk to the rebuilt ModelResponse, ensuring provider metadata
survives the chunk→response rebuild.
---------
Co-authored-by: naaa760 <neh6a683@gmail.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
OAuth tokens (sk-ant-oat*) require Authorization: Bearer header per
Anthropic's OAuth specification, but were being sent via x-api-key
which Anthropic rejects with 'invalid x-api-key'.
- optionally_handle_anthropic_oauth: detect OAuth tokens in api_key
param (standard chat flow), not just Authorization header
- get_anthropic_headers: use Authorization: Bearer + required OAuth
headers for OAuth tokens, x-api-key for regular API keys
- Passthrough messages: skip x-api-key when Authorization is set
- Add oauth-2025-04-20 to beta headers whitelist config
* Add http support to custom code guardrails + Unified guardrails for MCP + Agent guardrail support (#20619)
* fix: fix styling
* fix(custom_code_guardrail.py): add http support for custom code guardrails
allows users to call external guardrails on litellm with minimal code changes (no custom handlers)
Test guardrail integrations more easily
* feat(a2a/): add guardrails for agent interactions
allows the same guardrails for llm's to be applied to agents as well
* fix(a2a/): support passing guardrails to a2a from the UI
* style(code-editor): allow editing custom code guardrails on ui + add examples of pre/post calls for custom code guardrails
* feat(mcp/): support custom code guardrails for mcp calls
allows custom code guardrails to work on mcp input
* feat(chatui.tsx): support guardrails on mcp tool calls on playground
* fix(mypy): resolve missing return statements and type casting issues (#20618)
* fix(mypy): resolve missing return statements and type casting issues
* fix(pangea): use elif to prevent UnboundLocalError and handle None messages
Address Greptile review feedback:
- Make branches mutually exclusive using elif to prevent input_messages from being overwritten
- Handle case where data.get('messages') returns None to avoid passing invalid payload to Pangea API
---------
Co-authored-by: Shin <shin@openclaw.ai>
* [Feat] MCP Gateway - Allow setting MCP Servers as Private/Public available on Internet (#20607)
* update MCPAuthenticatedUser
* add available_on_public_internet for MCPs
* update claude.md
* init IPAddressUtils
* init available_on_public_internet
* add on REST endpoints
* filter with IP
* TestIsInternalIp
* _extract_mcp_headers_from_request
* init get_mcp_client_ip
* _get_general_settings
* allowed_server_ids
* address PR comments
* get_mcp_server_by_name fix
* fix server
* fix review comments
* get_public_mcp_servers
* address _get_allowed_mcp_servers
* fixing user_id
* [Feat] IP-Based Access Control for MCP Servers (#20620)
* update MCPAuthenticatedUser
* add available_on_public_internet for MCPs
* update claude.md
* init IPAddressUtils
* init available_on_public_internet
* add on REST endpoints
* filter with IP
* TestIsInternalIp
* _extract_mcp_headers_from_request
* init get_mcp_client_ip
* _get_general_settings
* allowed_server_ids
* address PR comments
* get_mcp_server_by_name fix
* fix server
* fix review comments
* get_public_mcp_servers
* address _get_allowed_mcp_servers
* test fix
* fix linting
* inint ui types
* add ui for managing MCP private/public
* add ui
* fixes
* add to schema
* add types
* fix endpoint
* add endpoint
* update manager
* test mcp
* dont use external party for ip address
* Add OpenAI/Azure release test suite with HTTP client lifecycle regression detection (#20622)
* docs (#20626)
* docs
* fix(mypy): resolve type checking errors in 5 files (#20627)
- a2a_protocol/exception_mapping_utils.py: Fix type ignore comment for None assignment
- caching/redis_cache.py: Add type ignore for async ping return type
- caching/redis_cluster_cache.py: Add type ignore for async ping return type
- llms/deprecated_providers/palm.py: Add type ignore for palm.generate_text
- proxy/auth/handle_jwt.py: Add type ignore for jwt.decode options argument
All changes add appropriate type: ignore comments to handle library typing inconsistencies.
* fix(test): update deprecated gemini embedding model (#20621)
Replace text-embedding-004 with gemini-embedding-001.
The old model was deprecated and returns 404:
'models/text-embedding-004 is not found for API version v1beta'
Co-authored-by: Shin <shin@openclaw.ai>
* ui new buil
* fix(http_handler): bypass cache when shared_session is provided for aiohttp tracing
When users pass a shared_session with trace_configs to acompletion(),
the get_async_httpx_client() function was ignoring it and returning
a cached client without the user's tracing configuration.
This fix bypasses the cache when shared_session is provided, ensuring
the user's ClientSession (with its trace_configs, connector settings, etc.)
is actually used for the request.
Fixes#20174
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Shin <shin@openclaw.ai>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
When an OpenAI-compatible upstream provider emits minimal streaming event
payloads that omit required fields (e.g. created_at, output, output_index,
content_index), Pydantic raises a ValidationError crashing the SSE stream
and returning HTTP 500.
Fall back to model_construct() on ValidationError, consistent with the
existing pattern in transform_response_api_response for non-streaming.
Fixes https://github.com/BerriAI/litellm/issues/20570
Signed-off-by: Varun Chawla <varun_6april@hotmail.com>
