Gemini 2.0+ natively accepts JSON Schema in tool parameters, including
bare {} (TYPE_UNSPECIFIED), anyOf with null, and lowercase types. The
existing _build_vertex_schema pipeline was coercing {} to {"type": "object"},
breaking JsonValue/Any field semantics (issue #22391).
Add _build_vertex_schema_for_gemini_2() that only resolves $ref (which
Gemini doesn't support in tools) and filters unsupported fields. Use it
for Gemini 2.0+ models, keeping the full transform for Gemini 1.5.
- Fix case-insensitive tool name matching in _tool_name_matches() so that
OpenAPI operationIds (camelCase) match lowercase registered tool names
when filtering by allowed_tools
- Fix get_base_url() to resolve relative server URLs (e.g. /api/v3) by
deriving full base URL from spec_path when OpenAPI spec has relative URLs
- Add tests for case-insensitive matching and filter_tools_by_allowed_tools
Made-with: Cursor
`all_models = user_api_key_dict.models` was creating an alias, so
`_get_models_from_access_groups` (which uses `.pop()`/`.extend()`) would
mutate the cached object in-place. Now both `.models` and `.team_models`
assignments create copies via `list()`.
Added test to verify the input is not mutated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds dedup to get_key_models and get_team_models to prevent duplicate
entries when access group member models overlap with proxy_model_list.
Removes dead assignment of all_models in get_team_models.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a team has "all-proxy-models", the model list expansion now includes
model access group names so they appear in the UI key creation form.
Also fixes get_key_models not forwarding include_model_access_groups to
_get_models_from_access_groups, and removes unused _unfurl_all_proxy_models.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix perform_redaction to handle dict representation of ModelResponse (from model_dump())
- Preserve full choices structure when redacting, redact content/audio in place
- Add _redact_standard_logging_object helper for standard_logging_object field
- Update test_logging_redaction_e2e_test assertions to expect choices format
- Add charity_engine to provider_endpoints_support.json
Fixes: test_standard_logging_payload, test_standard_logging_payload_audio
Made-with: Cursor
The SageMaker embedding handler was not using _load_credentials(),
which meant aws_role_name and aws_session_name parameters were
ignored. This prevented cross-account role assumption for embeddings
while it worked for completions.
Changes:
- Replace direct boto3 client creation with _load_credentials() call
- Create boto3.Session with assumed credentials
- Add comprehensive unit tests for role assumption
This aligns the embedding handler behavior with the completion handler,
which already supports role assumption via the BaseAWSLLM.get_credentials()
method.
Fixes cross-account SageMaker embedding access where users need to
assume a role in another account to invoke endpoints.
Any param in DEFAULT_CHAT_COMPLETION_PARAM_VALUES that arrives via
completion(**kwargs) is now automatically forwarded to
get_optional_params(), even if it's not a named parameter of
completion().
Previously, get_non_default_completion_params() excluded params in
OPENAI_CHAT_COMPLETION_PARAMS (assuming they'd be forwarded via the
named-param path), while optional_param_args only contained explicitly
named params. Params like 'store' that were in the known-params list
but not named params fell through both paths and were silently dropped.
The fix adds a 7-line loop after building optional_param_args that
forwards any kwargs present in DEFAULT_CHAT_COMPLETION_PARAM_VALUES.
This means new OpenAI params only need to be added to the constants
dict — no boilerplate changes to 3+ function signatures required.
Fixes#23087
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* feat(charity_engine): add Charity Engine provider
Charity Engine is a crowdsourced distributed computing platform that
donates processing power to charitable causes. Its inference API
provides OpenAI-compatible chat, completions, and embeddings endpoints.
* test(charity_engine): add provider config and resolution tests
Verify JSONProviderRegistry config, provider list membership,
model routing for charity_engine/<model>, and Router compatibility.
* feat(charity_engine): add Charity Engine to LlmProviders enum
Enables provider_list membership and LlmProviders.CHARITY_ENGINE
resolution required by the provider and test suite.
* fix(charity_engine): remove api_base_env to fix non-deterministic test
The CHARITY_ENGINE_API_BASE env var could override the base_url in CI,
causing test_charity_engine_provider_resolution to fail intermittently.
* fix(charity_engine): remove trailing slash from base_url
Claude's API returns assistant messages with empty text blocks
({"type": "text", "text": ""}) alongside tool_use blocks during
multi-turn tool-use conversations. These blocks are rejected when
sent back to the API with "text content blocks must be non-empty".
Sanitization already exists for other code paths (/v1/chat/completions
for both Anthropic and Bedrock), but NOT for the /v1/messages native
path. This adds the same treatment by stripping empty text blocks
from messages in async_anthropic_messages_handler before they are
forwarded to the provider.
Fixes#22930
_get_openai_compatible_provider_info already returns an api_base
ending in /v1, but get_models prepended another /v1, producing
.../inference/v1/v1/accounts/... which 404s.
Strip the trailing /v1 from api_base before re-adding it so that
both the default and any user-supplied base work correctly.
Add parametrized tests covering the default URL, trailing-slash,
custom base with /v1, and custom base without /v1.
Fixes#23106
The bug occurred when user data inadvertently contained reserved Python
keywords like 'self', 'params', or '__class__' as keys. When such a dict
was unpacked via **kwargs to LiteLLM_Params() or GenericLiteLLMParams(),
Python raised TypeError because 'self' was passed both implicitly and
as a keyword argument.
The fix:
- Add a Pydantic model_validator(mode='before') to GenericLiteLLMParams
that filters out reserved keys ('self', 'params', '__class__') before
validation
- Move the max_retries str-to-int conversion into the same validator
- Remove the custom __init__ methods from both GenericLiteLLMParams and
LiteLLM_Params, since the validator now handles the preprocessing
- Clean up unused VERTEX_CREDENTIALS_TYPES import
This fix applies to all classes that inherit from GenericLiteLLMParams,
including LiteLLM_Params and updateLiteLLMParams.
Added comprehensive tests in tests/test_litellm/test_litellm_params_reserved_keys.py
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
The test calls OpenAI's gpt-4o-audio-preview model which sometimes
doesn't return usage data in the streaming response. Fixed by:
- Adding @pytest.mark.flaky(retries=5, delay=2) for retry handling
- Fixing usage_obj loop to check chunk.usage is not None
- Skipping gracefully when OpenAI doesn't return usage data
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
