mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
39e0efa11d
* fix(anthropic-messages): send bare-string tool_choice to Responses API, propagate router-alias litellm_params
The Anthropic /v1/messages -> Responses API adapter always wrapped
tool_choice in an object ({"type": "auto"}, {"type": "required"}), but
the Responses API's tool_choice schema for these cases is a bare
string ("auto"/"required"/"none"). Sending the object shape to an
OpenAI-compatible backend (e.g. vLLM) fails Pydantic validation with a
400. The "none" case also fell through to "auto" instead of mapping to
"none".
Separately, litellm_params configured directly on a router-alias
deployment (auto_router/complexity_router, adaptive_router,
quality_router, or semantic auto_router) - e.g.
cache_control_injection_points, drop_params - were silently dropped
for every request through that alias. async_pre_routing_hook swaps
`model` from the alias name to the selected tier/route's model before
the deployment lookup runs, so the outbound call only ever merged in
the tier deployment's own litellm_params, never the alias's. Register
non-routing-config litellm_params from the alias deployment and apply
them to the request whenever a pre-routing hook substitutes the model.
* fix: satisfy ruff-strict-budget UP006 and router coverage checker
Use builtin dict[...] generics instead of typing.Dict for the two new
annotations introduced in the previous commit, since they pushed
UP006 over the codebase ceiling in ruff-strict-budget.json. Add a
direct unit test for _register_pre_routing_alias_overrides so the
text-based router_code_coverage.py checker sees it exercised by name.
* fix(router): replace alias-param denylist with a tight allowlist
_PRE_ROUTING_ALIAS_RESERVED_PARAMS excluded router-init-only keys from
the alias's litellm_params before forwarding the rest as request
kwargs, but GenericLiteLLMParams also holds deployment-management
fields (tpm, rpm, weight, tags, max_budget, budget_duration,
use_in_pass_through, litellm_credential_name, ...) on the same object.
Any of those left off the denylist would get silently forwarded as if
they were request kwargs.
Replace the denylist with a tight allowlist of exactly the two
request-shaping params this feature exists for - drop_params and
cache_control_injection_points - so unrelated management fields never
reach the outbound call regardless of what else GenericLiteLLMParams
grows to hold.
* fix(router): re-register adaptive-alias overrides on set_model_list reload
set_model_list() unconditionally clears pre_routing_alias_overrides on
every call (e.g. /config/reload), but _finalize_adaptive_router_if_configured()
skips rebuilding an AdaptiveRouter whose model_name already exists in
self.adaptive_routers - so _register_pre_routing_alias_overrides() never
ran again for an auto_router/adaptive_router alias after a reload,
silently dropping its drop_params/cache_control_injection_points.
Build the Deployment unconditionally and re-register its overrides even
on the skip-existing-router path; only the (expensive) AdaptiveRouter
construction itself stays skipped.
* style: ruff format after merging litellm_internal_staging
* fix(router): drop the alias-param allowlist, exclude only model
Per review discussion: instead of a router.py-local allowlist of exactly
which litellm_params an alias (auto_router/complexity_router,
adaptive_router, quality_router, semantic auto_router) can forward to
the request it routes, _register_pre_routing_alias_overrides now
forwards everything except `model` (the alias marker itself, e.g.
auto_router/complexity_router, never a real provider model).
Router-init-only fields (complexity_router_config,
complexity_router_default_model, auto_router_config,
auto_router_config_path, auto_router_default_model,
auto_router_embedding_model, adaptive_router_config,
adaptive_router_default_model, quality_router_config,
quality_router_default_model) now flow into request_kwargs unfiltered
too. That's safe because litellm.completion()/acompletion() already
strips anything in litellm.types.utils.all_litellm_params before
building the provider request - added these 10 keys there, alongside
the deployment-management fields (tpm, rpm, weight, ...) already listed.
Verified live: without that addition, complexity_router_config lands in
extra_body and ships raw to the provider; with it, it's stripped.
This moves the "which fields aren't real LLM params" list from a
router.py-local allowlist to the single existing global list every
completion() call already depends on, instead of maintaining two.
* refactor(router): look up alias litellm_params on demand instead of caching them
_register_pre_routing_alias_overrides cached each alias's litellm_params
into self.pre_routing_alias_overrides at deployment-init time, which
required keeping that cache in sync with set_model_list() reloads - the
exact bug the previous adaptive-router-reload fix was patching around
(AdaptiveRouter survives a reload, but the cache didn't always get
refreshed to match).
Delete the cache and the registration method entirely. async_pre_routing_hook
now looks up the alias's own litellm_params directly from self.model_list
via self.model_name_to_deployment_indices at request time, the same
model_list that's already correctly rebuilt on every set_model_list()
call. No second piece of state to invalidate, so the reload staleness
bug class isn't possible anymore, and it's less code than before.
|
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| integration | ||
| litellm | ||
| litellm-proxy-extras | ||
| litellm_core_utils | ||
| litellm_utils_tests | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| old_proxy_tests/tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_behavior | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_migration_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| scim_tests | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_litellm | ||
| unified_google_tests | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _openai_record_replay_proxy.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _ws_vcr.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_anthropic_compaction_usage.py | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_config.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_entrypoint.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_passthrough_endpoints.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.