mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
b76a858826
* feat: declarative fallback generalizations for unknown models Unknown or newly-released models previously degraded (missed cost lookups, wrong supports_* flags, broken provider routing) and were patched with one-off hardcoded regexes scattered across Python. This adds a single data-driven source of truth: a fallback_generalizations block in model_prices_and_context_window.json holding ordered, case-insensitive regex rules that map a model name to the metadata to apply when it has no exact entry. A new fallback_generalizations module owns the rules and a compiled-regex cache that is built once and invalidated on reload, so the O(n) scan runs only on a cache miss. get_llm_provider now routes an otherwise-unknown model via the first matching rule's litellm_provider, replacing the hardcoded _CLAUDE_PATTERN and _matches_claude_model_pattern. _get_model_info_helper falls back to a matching rule's model_info after the exact lookups miss, so get_model_info and the supports_* helpers resolve unknown models from the same rule. get_model_cost_map extracts the block out of the returned map, and the integrity check now counts real model entries (excluding reserved meta keys) so the new key cannot mask a genuinely shrunk upstream file. The top level of the file stays a flat map of models so existing litellm releases that fetch the live file keep working and keep receiving updates; the block ships in both the root file and the bundled backup. An anthropic-claude rule reproduces the old future-claude routing and additionally supplies capability flags and a context window https://claude.ai/code/session_01G8Jro8dPLktwnaaSJwVDpo * refactor(anthropic): derive adaptive-thinking from a version threshold; harden generalizations Replace the per-minor-version _is_claude_4_6_model / _is_claude_4_7_model substring matchers with a single _claude_version_at_least predicate that parses the Claude family version from the model name and compares against 4.6. This covers 4.8/4.9/5.x without a code change (the old matchers missed 4.8 entirely) while keeping an explicit supports_adaptive_thinking flag authoritative when present, so there is one source of truth. The two direct call sites in the chat transformation now route through _is_adaptive_thinking_model instead of the deleted matchers. Also address review feedback on the generalizations module: return a copy of the matched model_info so a future caller cannot mutate the compiled-rule cache, document that patterns are matched with re.search and must anchor with ^ and $, and reindent the fallback_generalizations block to the file's 2-space style in both JSON files. https://claude.ai/code/session_01G8Jro8dPLktwnaaSJwVDpo * fix(anthropic): surface adaptive-thinking from the cost map; fix date misparse supports_adaptive_thinking shipped in the model cost map but was never declared on ModelInfo nor copied during construction, so get_model_info (and the supports_* factory) silently dropped it for every provider-prefixed or generalized name; only a bare base entry resolved. Wire it through ModelInfo like the other capability flags and backfill the flag onto the genuine Claude 4.6/4.7/4.8 entries across providers so the data, not code, declares the capability. The anthropic-claude fallback rule also carries the flag (and now accepts a dotted minor, e.g. 4.6) so an unmapped future Claude degrades to adaptive thinking without a code change. Tighten the Claude version parser so an eight-digit date suffix (claude-opus-4-20250514, the non-adaptive Opus 4.0) is no longer read as minor 4.20250514. The cost map stays authoritative; the version check is only a fallback for provider-prefixed names (bedrock/invoke routes, -v1-less ids) that resolve to no mapped entry and so cannot be reached by an exact lookup or the bare-name rule. https://claude.ai/code/session_01G8Jro8dPLktwnaaSJwVDpo * fix(anthropic): date-safe adaptive-thinking version fallback, conservative fallback pricing, ruff strict gate Reconcile adaptive-thinking detection after merging litellm_internal_staging. Keep the cost-map resolver (_supports_model_capability) as the source of truth and add a date-safe opus/sonnet/haiku >= 4.6 name version as a fallback for provider-prefixed ids the cost map cannot resolve (e.g. bedrock/invoke/us.anthropic.claude-opus-4-6). A two-digit cap on the minor keeps an eight-digit date suffix from being misread as a minor version, so the dated Claude 4.0 release stays non-adaptive Price the shipped anthropic-claude fallback rule at the Opus tier so an unknown or newly released Claude is over-costed rather than billed as free Drop the module-level global state in fallback_generalizations (PLW0603) in favor of a small registry object, and switch its annotations plus the new utils helper to builtin generics (UP006), bringing the ruff strict-rule totals back under ceiling * refactor(anthropic): drive adaptive-thinking version gate from a declarative rule Replace the bespoke _claude_version_at_least heuristic with a version-gated fallback_generalizations rule. Unmapped Claude ids now resolve adaptive thinking purely from the cost map: an explicit entry, or the new self-contained anthropic-claude-adaptive-thinking rule that matches opus/sonnet/haiku >= 4.6 (covering 5.x, 6.x and beyond with no code change). New families ship via Price Data Reload instead of a code edit The rule carries the same Opus-tier pricing as the broad anthropic-claude rule plus supports_adaptive_thinking, and is matched first; the broad rule stays version-neutral, so an unmapped >= 4.6 Claude resolves to full pricing and the adaptive flag from one rule, while a sub-4.6 alias such as claude-opus-4-0 is still priced yet stays non-adaptive. The regex caps the minor at two digits so a dated 4.0 id (...-4-20250514) is never read as a >= 4.6 minor * refactor(anthropic): dedupe adaptive-thinking rule via declarative extends The version-gated anthropic-claude-adaptive-thinking rule duplicated the broad anthropic-claude rule's entire Opus-tier price block because rules do not merge: first match wins and returns one rule's whole model_info, so the adaptive rule had to be self-contained. Add a declarative extends field to fallback_generalizations: a rule names a parent and inherits its model_info, with its own keys overriding. Inheritance is resolved once at install time against each rule's raw model_info, so the adaptive rule now carries only its delta (supports_adaptive_thinking) and inherits pricing from the broad rule. Runtime matching, provider routing and gating are unchanged; the broad rule stays anchored and first-match-wins still holds. * docs(anthropic): add ignored description key documenting each generalization regex * fix(anthropic): drop fabricated pricing from the anthropic-claude fallback rule Per review feedback, the base rule no longer carries input/output/cache costs, and the adaptive-thinking rule that extends it inherits that no-pricing model_info. Pricing an unmapped model at a guessed tier reports a confidently-wrong cost without the caller knowing; dropping it keeps the standard unpriced behavior (zero, not a fabricated number) so a missing price stays visible. The rules still supply provider routing, context window, and capability flags, so a brand-new Claude can still be called and its capabilities (including adaptive thinking for >= 4.6) resolved. Description and tests updated to match |
||
|---|---|---|
| .. | ||
| .litellm_cache | ||
| auto_router | ||
| example_config_yaml | ||
| test_configs | ||
| test_model_response_typing | ||
| azure_fine_tune.jsonl | ||
| azure_speech.mp3 | ||
| batch_job_results_furniture.jsonl | ||
| cache_unit_tests.py | ||
| conftest.py | ||
| create_mock_standard_logging_payload.py | ||
| data_map.txt | ||
| eagle.wav | ||
| example.jsonl | ||
| gettysburg.wav | ||
| large_text.py | ||
| model_cost.json | ||
| openai_batch_completions.jsonl | ||
| openai_batch_completions_router.jsonl | ||
| speech_vertex.mp3 | ||
| stream_chunk_testdata.py | ||
| test_acompletion.py | ||
| test_acompletion_fallbacks.py | ||
| test_acooldowns_router.py | ||
| test_add_function_to_prompt.py | ||
| test_add_update_models.py | ||
| test_aim_guardrails.py | ||
| test_alangfuse.py | ||
| test_amazing_vertex_completion.py | ||
| test_anthropic_prompt_caching.py | ||
| test_arize_ai.py | ||
| test_arize_phoenix.py | ||
| test_assistants.py | ||
| test_async_fn.py | ||
| test_auth_utils.py | ||
| test_azure_anthropic_sync_post.py | ||
| test_azure_content_safety.py | ||
| test_azure_openai.py | ||
| test_azure_perf.py | ||
| test_basic_python_version.py | ||
| test_batch_completion_return_exceptions.py | ||
| test_batch_completions.py | ||
| test_blocked_user_list.py | ||
| test_braintrust.py | ||
| test_budget_manager.py | ||
| test_cache_preset_key.py | ||
| test_caching.py | ||
| test_caching_handler.py | ||
| test_caching_ssl.py | ||
| test_class.py | ||
| test_completion.py | ||
| test_completion_cost.py | ||
| test_completion_with_retries.py | ||
| test_config.py | ||
| test_cost_calc.py | ||
| test_custom_api_logger.py | ||
| test_custom_callback_input.py | ||
| test_custom_llm.py | ||
| test_custom_logger.py | ||
| test_disk_cache_unit_tests.py | ||
| test_docker_no_network_on_deploy.py | ||
| test_dual_cache.py | ||
| test_dynamic_rate_limit_handler.py | ||
| test_dynamodb_logs.py | ||
| test_embedding.py | ||
| test_exceptions.py | ||
| test_fake_openai_endpoint.py | ||
| test_file_types.py | ||
| test_function_call_parsing.py | ||
| test_function_calling.py | ||
| test_function_setup.py | ||
| test_gcs_bucket.py | ||
| test_gcs_cache_unit_tests.py | ||
| test_gemini_reasoning_content.py | ||
| test_get_llm_provider.py | ||
| test_get_model_file.py | ||
| test_get_model_info.py | ||
| test_get_optional_params_embeddings.py | ||
| test_get_optional_params_functions_not_supported.py | ||
| test_google_ai_studio_gemini.py | ||
| test_guardrails_ai.py | ||
| test_helicone_integration.py | ||
| test_http_parsing_utils.py | ||
| test_img_resize.py | ||
| test_lakera_ai_prompt_injection.py | ||
| test_langchain_ChatLiteLLM.py | ||
| test_langsmith.py | ||
| test_least_busy_routing.py | ||
| test_litellm_max_budget.py | ||
| test_llm_guard.py | ||
| test_load_test_router_s3.py | ||
| test_loadtest_router.py | ||
| test_logfire.py | ||
| test_logging.py | ||
| test_longer_context_fallback.py | ||
| test_lowest_cost_routing.py | ||
| test_lowest_latency_routing.py | ||
| test_lunary.py | ||
| test_max_tpm_rpm_limiter.py | ||
| test_mem_leak.py | ||
| test_mem_usage.py | ||
| test_mock_request.py | ||
| test_model_alias_map.py | ||
| test_model_max_token_adjust.py | ||
| test_multiple_deployments.py | ||
| test_no_top_level_test_invocations.py | ||
| test_ollama.py | ||
| test_ollama_local.py | ||
| test_ollama_local_chat.py | ||
| test_openai_moderations_hook.py | ||
| test_opik.py | ||
| test_pass_through_endpoints.py | ||
| test_profiling_router.py | ||
| test_prometheus_service.py | ||
| test_prompt_caching.py | ||
| test_prompt_injection_detection.py | ||
| test_promptlayer_integration.py | ||
| test_provider_specific_config.py | ||
| test_pydantic.py | ||
| test_pydantic_namespaces.py | ||
| test_redis_batch_optimizations.py | ||
| test_register_model.py | ||
| test_responses_stream_cache_keys.py | ||
| test_router.py | ||
| test_router_auto_router.py | ||
| test_router_batch_completion.py | ||
| test_router_budget_limiter.py | ||
| test_router_caching.py | ||
| test_router_client_init.py | ||
| test_router_cooldown_handlers.py | ||
| test_router_custom_routing.py | ||
| test_router_debug_logs.py | ||
| test_router_fallback_handlers.py | ||
| test_router_fallbacks.py | ||
| test_router_get_deployments.py | ||
| test_router_max_parallel_requests.py | ||
| test_router_pattern_matching.py | ||
| test_router_retries.py | ||
| test_router_timeout.py | ||
| test_router_utils.py | ||
| test_router_with_fallbacks.py | ||
| test_rules.py | ||
| test_sagemaker.py | ||
| test_sagemaker_nova_integration.py | ||
| test_scheduler.py | ||
| test_secret_detect_hook.py | ||
| test_spend_calculate_endpoint.py | ||
| test_stream_chunk_builder.py | ||
| test_streaming.py | ||
| test_supabase_integration.py | ||
| test_team_config.py | ||
| test_text_completion.py | ||
| test_timeout.py | ||
| test_together_ai.py | ||
| test_tpm_rpm_routing_v2.py | ||
| test_traceloop.py | ||
| test_ui_sso_helper_utils.py | ||
| test_unit_test_caching.py | ||
| test_update_spend.py | ||
| test_validate_environment.py | ||
| test_wandb.py | ||
| user_cost.json | ||
| vertex_ai.jsonl | ||
| vertex_batch_completions.jsonl | ||
| vertex_key.json | ||
| whitelisted_bedrock_models.txt | ||