When a fetch fails, the button now exits the loading state instead of
staying stuck on "Fetching" indefinitely.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Show a Fetch/Fetching button next to "Showing X of Y results" that acts as
both a manual refetch trigger and a loading indicator. The "Loading keys..."
message now only appears on initial load; subsequent refetches keep the table
visible with stale data (via React Query's keepPreviousData).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix case-insensitive tool name matching in _tool_name_matches() so that
OpenAPI operationIds (camelCase) match lowercase registered tool names
when filtering by allowed_tools
- Fix get_base_url() to resolve relative server URLs (e.g. /api/v3) by
deriving full base URL from spec_path when OpenAPI spec has relative URLs
- Add tests for case-insensitive matching and filter_tools_by_allowed_tools
Made-with: Cursor
`all_models = user_api_key_dict.models` was creating an alias, so
`_get_models_from_access_groups` (which uses `.pop()`/`.extend()`) would
mutate the cached object in-place. Now both `.models` and `.team_models`
assignments create copies via `list()`.
Added test to verify the input is not mutated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds dedup to get_key_models and get_team_models to prevent duplicate
entries when access group member models overlap with proxy_model_list.
Removes dead assignment of all_models in get_team_models.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a team has "all-proxy-models", the model list expansion now includes
model access group names so they appear in the UI key creation form.
Also fixes get_key_models not forwarding include_model_access_groups to
_get_models_from_access_groups, and removes unused _unfurl_all_proxy_models.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix perform_redaction to handle dict representation of ModelResponse (from model_dump())
- Preserve full choices structure when redacting, redact content/audio in place
- Add _redact_standard_logging_object helper for standard_logging_object field
- Update test_logging_redaction_e2e_test assertions to expect choices format
- Add charity_engine to provider_endpoints_support.json
Fixes: test_standard_logging_payload, test_standard_logging_payload_audio
Made-with: Cursor
Extend the "Proxy database access" section with guidelines to prevent
common DB performance issues, tailored to actual Prisma usage patterns
in the litellm codebase: N+1 queries, client-side processing, batching
writes, bounding result sets, select on wide tables, index coverage,
and schema file sync.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Any param in DEFAULT_CHAT_COMPLETION_PARAM_VALUES that arrives via
completion(**kwargs) is now automatically forwarded to
get_optional_params(), even if it's not a named parameter of
completion().
Previously, get_non_default_completion_params() excluded params in
OPENAI_CHAT_COMPLETION_PARAMS (assuming they'd be forwarded via the
named-param path), while optional_param_args only contained explicitly
named params. Params like 'store' that were in the known-params list
but not named params fell through both paths and were silently dropped.
The fix adds a 7-line loop after building optional_param_args that
forwards any kwargs present in DEFAULT_CHAT_COMPLETION_PARAM_VALUES.
This means new OpenAI params only need to be added to the constants
dict — no boilerplate changes to 3+ function signatures required.
Fixes#23087
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
The SageMaker embedding handler was not using _load_credentials(),
which meant aws_role_name and aws_session_name parameters were
ignored. This prevented cross-account role assumption for embeddings
while it worked for completions.
Changes:
- Replace direct boto3 client creation with _load_credentials() call
- Create boto3.Session with assumed credentials
- Add comprehensive unit tests for role assumption
This aligns the embedding handler behavior with the completion handler,
which already supports role assumption via the BaseAWSLLM.get_credentials()
method.
Fixes cross-account SageMaker embedding access where users need to
assume a role in another account to invoke endpoints.
Any param in DEFAULT_CHAT_COMPLETION_PARAM_VALUES that arrives via
completion(**kwargs) is now automatically forwarded to
get_optional_params(), even if it's not a named parameter of
completion().
Previously, get_non_default_completion_params() excluded params in
OPENAI_CHAT_COMPLETION_PARAMS (assuming they'd be forwarded via the
named-param path), while optional_param_args only contained explicitly
named params. Params like 'store' that were in the known-params list
but not named params fell through both paths and were silently dropped.
The fix adds a 7-line loop after building optional_param_args that
forwards any kwargs present in DEFAULT_CHAT_COMPLETION_PARAM_VALUES.
This means new OpenAI params only need to be added to the constants
dict — no boilerplate changes to 3+ function signatures required.
Fixes#23087
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* feat(charity_engine): add Charity Engine provider
Charity Engine is a crowdsourced distributed computing platform that
donates processing power to charitable causes. Its inference API
provides OpenAI-compatible chat, completions, and embeddings endpoints.
* test(charity_engine): add provider config and resolution tests
Verify JSONProviderRegistry config, provider list membership,
model routing for charity_engine/<model>, and Router compatibility.
* feat(charity_engine): add Charity Engine to LlmProviders enum
Enables provider_list membership and LlmProviders.CHARITY_ENGINE
resolution required by the provider and test suite.
* fix(charity_engine): remove api_base_env to fix non-deterministic test
The CHARITY_ENGINE_API_BASE env var could override the base_url in CI,
causing test_charity_engine_provider_resolution to fail intermittently.
* fix(charity_engine): remove trailing slash from base_url
Claude's API returns assistant messages with empty text blocks
({"type": "text", "text": ""}) alongside tool_use blocks during
multi-turn tool-use conversations. These blocks are rejected when
sent back to the API with "text content blocks must be non-empty".
Sanitization already exists for other code paths (/v1/chat/completions
for both Anthropic and Bedrock), but NOT for the /v1/messages native
path. This adds the same treatment by stripping empty text blocks
from messages in async_anthropic_messages_handler before they are
forwarded to the provider.
Fixes#22930
_get_openai_compatible_provider_info already returns an api_base
ending in /v1, but get_models prepended another /v1, producing
.../inference/v1/v1/accounts/... which 404s.
Strip the trailing /v1 from api_base before re-adding it so that
both the default and any user-supplied base work correctly.
Add parametrized tests covering the default URL, trailing-slash,
custom base with /v1, and custom base without /v1.
Fixes#23106
In _get_available_deployments, when streaming mode is active the code
sums time-to-first-token values from item_ttft_latency but divides by
len(item_latency) instead of len(item_ttft_latency). These lists can
have different lengths, producing an incorrect average that skews
lowest-latency routing decisions for streaming requests.
Signed-off-by: JiangNan <1394485448@qq.com>
When value > 1 (e.g., "2mo") and current_month + value > 12,
target_month exceeds valid range (1-12), causing ValueError in
datetime constructor. For example, calling duration_in_seconds("2mo")
in November produces target_month=13.
Use modular arithmetic to correctly wrap months and increment year.
Signed-off-by: JiangNan <1394485448@qq.com>
* add new azure gpt models
* add versionless azure/gpt-5.4 models
* Undated azure/gpt-5.4 alias missing supports_service_tier
* indicate service tier support for azure/gpt-5.3-chat
* fix priority tier pricing for new azure/gpt models
The bug occurred when user data inadvertently contained reserved Python
keywords like 'self', 'params', or '__class__' as keys. When such a dict
was unpacked via **kwargs to LiteLLM_Params() or GenericLiteLLMParams(),
Python raised TypeError because 'self' was passed both implicitly and
as a keyword argument.
The fix:
- Add a Pydantic model_validator(mode='before') to GenericLiteLLMParams
that filters out reserved keys ('self', 'params', '__class__') before
validation
- Move the max_retries str-to-int conversion into the same validator
- Remove the custom __init__ methods from both GenericLiteLLMParams and
LiteLLM_Params, since the validator now handles the preprocessing
- Clean up unused VERTEX_CREDENTIALS_TYPES import
This fix applies to all classes that inherit from GenericLiteLLMParams,
including LiteLLM_Params and updateLiteLLMParams.
Added comprehensive tests in tests/test_litellm/test_litellm_params_reserved_keys.py
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Replace Tremor Button components with antd Button in the user
management flow (Invite User, Bulk Invite Users, Select Users,
Bulk Edit, and onboarding modal copy link button).
The test calls OpenAI's gpt-4o-audio-preview model which sometimes
doesn't return usage data in the streaming response. Fixed by:
- Adding @pytest.mark.flaky(retries=5, delay=2) for retry handling
- Fixing usage_obj loop to check chunk.usage is not None
- Skipping gracefully when OpenAI doesn't return usage data
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test_create_skill test was consistently failing in CI with a 500 from
Anthropic because the SKILL.md frontmatter always used the same hardcoded
name (test-skill-litellm). Since test_delete_skill is permanently skipped,
skills accumulate in the CI account, and re-creating with a duplicate name
triggers an Internal Server Error on Anthropic's side.
Fix: pass a timestamp-based unique_suffix to create_skill_zip so each run
produces a distinct skill name in the zip's SKILL.md frontmatter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add input_fidelity parameter ("high"/"low") to the image edit pipeline,
allowing users to control how much effort the model exerts to match
input image style and features. Fixes#22813.
The tag daily activity endpoint used compute_tag_metadata_totals which
deduplicates by request_id, but LiteLLM_DailyTagSpend stores aggregated
daily records where request_id is either NULL or stale. This caused all
metadata totals (total_spend, total_requests, etc.) to be 0 in the UI
cost panel. Now uses the same standard totals as every other entity type.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
