mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-11 03:38:38 +00:00
8b1990b4bc
* feat(decisions): add unified /v1/decisions endpoint for Jev-compatible providers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(decisions): register typesafe as a provider so Jev deployments load Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(decisions): move provider endpoints under llms and validate proxy bodies Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(decisions): add Cloudflare Clef and Strands Decider backends Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(decisions): register decisions routes for managed agents and gateway Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(decisions): use raw regex for cloudflare missing account match Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(decisions): avoid cast in Cloudflare response unwrapping Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(decisions): default model, evaluation health probe, short Cloudflare names The proxy validates only state and questions, so a request without a model falls through to the configured default model like every other route. Health checks probe evaluation-mode deployments through the Decisions API instead of failing with an unsupported mode, and cloudflare/clef and cloudflare/clef-flash get cost-map rows so the short names resolve a mode and a price. The registry no longer claims typed decisions for a provider with no backend. * fix(decisions): let health_check_params override the evaluation probe Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): audit the decisions endpoint across providers, limits, health and chaos Adds the /v1/decisions audit cells: one wire contract per provider (path, key, body and cost-map billing), the gateway-only fields and tags, the sad paths (invalid bodies, unknown model, key checks, api_base in the body, upstream 401/429/500, a 200 without answers, an unreachable upstream), the two evaluation-mode health probes, and three chaos cells (a mixed-failure burst over both routes, a worker SIGKILL mid-burst, an upstream outage and restart on the same port). The PR's cost case read the upstream observations through the gateway, which answers 404 for that path; it now reads them from the upstream URL. The owned proxy harness takes extra CLI arguments, and its graceful stop waits as long as a worker boot may take, since a worker still starting honors SIGTERM only once it is up and the 30 second wait forced a cleanup under load. * fix(decisions): send env API keys to a configured api_base Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(decisions): add zero-cost evaluation cost-map entry for Strands Decider Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(decisions): register the routes through the lazy feature registry The Decisions router was included at import, ahead of the config and DB pass-through endpoints, so a pass-through configured at /v1/decisions was skipped and answered 400 as an unknown Decisions provider. The routes now register through LAZY_FEATURES, which splices them in after every eager route, so a pass-through at /v1/decisions keeps its route while /decisions still serves natively. The lazy OpenAPI snapshot carries the two paths so the schema shows them before the first call. The audit cells add the env-key egress to a configured api_base, the client api_base opt-in shared with chat, the pass-through precedence on an owned proxy, and the Strands evaluation health check resolved from the cost map. The integration config exports the Perplexity env key the first cell needs. * fix(decisions): keep the Cloudflare api_base message in its transformation and read the audit upstream once per cell * fix(proxy): let a config pass-through beat a lazily registered route in eager mode With LITELLM_DISABLE_LAZY_ROUTES set the decisions routes are registered at startup, so SafeRouteAdder treated a config pass-through at exactly /v1/decisions as already registered and dropped it. In lazy mode a pass-through created through the API after the first native call was skipped the same way. Routes a lazy feature owns no longer count as registered, and a route added at one of their paths is placed ahead of them, the precedence lazy mode gives a config pass-through when the feature has not loaded yet. --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| audio_utils | ||
| llm_cost_calc | ||
| llm_response_utils | ||
| prompt_templates | ||
| specialty_caches | ||
| __init__.py | ||
| conftest.py | ||
| event_loop_lag.py | ||
| fake_secret_vault.py | ||
| messages_with_counts.py | ||
| test_agentic_followup_kwargs.py | ||
| test_anthropic_dedup_factory.py | ||
| test_api_route_to_call_types.py | ||
| test_audio_utils.py | ||
| test_aws_partition.py | ||
| test_bedrock_converse_dedup_factory.py | ||
| test_bug_report.py | ||
| test_chat_completion_agentic_loop.py | ||
| test_classifier_logging.py | ||
| test_cli_token_utils.py | ||
| test_cloud_storage_security.py | ||
| test_codestral_provider_routing.py | ||
| test_core_helpers.py | ||
| test_coroutine_checker.py | ||
| test_dd_tracing.py | ||
| test_decode_special_tokens.py | ||
| test_dot_notation_indexing.py | ||
| test_duration_parser.py | ||
| test_error_normalization.py | ||
| test_exception_mapping_utils.py | ||
| test_extract_base64_image.py | ||
| test_fallback_generalizations.py | ||
| test_fallback_utils.py | ||
| test_get_litellm_params.py | ||
| test_get_llm_provider_endpoint_match.py | ||
| test_get_llm_provider_logic.py | ||
| test_get_model_cost_map.py | ||
| test_get_supported_openai_params.py | ||
| test_health_check_helpers.py | ||
| test_image_handling.py | ||
| test_initialize_dynamic_callback_params.py | ||
| test_internal_call_metadata.py | ||
| test_json_fragment_accumulator.py | ||
| test_json_schema_validation.py | ||
| test_litellm_logging.py | ||
| test_llm_judge.py | ||
| test_llm_request_utils.py | ||
| test_logging_utils.py | ||
| test_logging_worker.py | ||
| test_max_streaming_duration.py | ||
| test_model_param_helper.py | ||
| test_model_response_utils.py | ||
| test_private_json.py | ||
| test_provider_affinity.py | ||
| test_provider_specific_headers.py | ||
| test_ptu_pricing.py | ||
| test_realtime_errors.py | ||
| test_realtime_streaming.py | ||
| test_redact_messages.py | ||
| test_request_timeout_resolver.py | ||
| test_retry_after_headers.py | ||
| test_safe_divide_seconds.py | ||
| test_safe_json_dumps.py | ||
| test_sensitive_data_masker.py | ||
| test_sentry_scrubbing.py | ||
| test_served_output_texts.py | ||
| test_streaming_chunk_builder_cursor.py | ||
| test_streaming_chunk_builder_server_tool_use.py | ||
| test_streaming_chunk_builder_utils.py | ||
| test_streaming_handler.py | ||
| test_streaming_overhead.py | ||
| test_thread_pool_executor.py | ||
| test_token_counter.py | ||
| test_token_counter_tool.py | ||
| test_token_counter_tool_data.py | ||
| test_tokenizer.py | ||
| test_tool_search_spend_logging.py | ||
| test_url_utils.py | ||
| test_xai_oauth_routing.py | ||