mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-16 04:29:31 +00:00
fd84e97f49
* feat(providers): add Inference APIs as OpenAI-compatible provider Registers https://api.inferenceapis.com/v1 as a JSON-configured provider (chat completions and Responses API). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(providers): list Inference APIs in Add Model and the provider docs index, add request-level tests - provider_create_fields.json entry so the Add Model dropdown test passes - provider_endpoints_support.json (+ backup) entry so the provider docs check passes - tests now capture the outgoing HTTP request (URL, auth header, body) instead of only inspecting registrations - drop a redundant comment Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(providers): inject a recording HTTP session instead of patching httpx.Client Request-level tests now set litellm.client_session to an httpx.Client with a MockTransport, so nothing is monkeypatched on a class. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qUBY2P8qTTuJCUeMzTCD4 * chore(providers): drop redundant comment on the inferenceapis entry Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qUBY2P8qTTuJCUeMzTCD4 * feat(inferenceapis): add cost map rows for the 12 chat models and test the responses route Prices and context windows come from https://inferenceapis.com/pricing and GET /v1/models. Moves the provider test to tests/unit and adds the interactions flag the endpoint matrix now carries. * feat(inferenceapis): price the vendor's chat model aliases and mark /v1/messages supported * fix(inferenceapis): serve /v1/messages natively and keep audio routes off the provider * fix(json providers): send provider-specific params in extra_body and report context overflow as ContextWindowExceededError * fix(exception mapping): match the Inference APIs overflow phrase instead of any context_length_exceeded substring * fix(exception mapping): map context overflow to ContextWindowExceededError for Inference APIs only Providers without their own mapper keep reporting an overflow as BadRequestError, since the router raises a ContextWindowExceededError instead of trying generic fallbacks when context_window_fallbacks has no entry for the group. Adds the integration cells for the provider, its fallbacks, and the extra_body and overflow families. * test(inferenceapis): hold upstream replies across the worker kill in the chaos cell A reply the killed worker sent before SIGKILL can lose its queued spend row, which the product never promised to keep. Holding the upstream until after the kill means only the surviving worker answers, so every 200 must log exactly once. * test(inferenceapis): make the chaos cell kill the quieter worker and always release held replies The survivor now must hold at least one in-flight request, so the burst check cannot pass as 0 == 0, and the held vendor is released on exit so a failed assertion cannot stall the wire server teardown --------- Co-authored-by: TaylorHawkes <857858+TaylorHawkes@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| conftest.py | ||
| test_aiohttp_session_rebuild_wire.py | ||
| test_anthropic_messages_bridge_mid_stream_failure_sdk.py | ||
| test_azure_prompt_shield_tuple_messages.py | ||
| test_bedrock_converse_redacted_thinking_router_sdk.py | ||
| test_bedrock_converse_stream_sync_decoder_wire.py | ||
| test_bedrock_model_discovery_sdk.py | ||
| test_cache_control_census_pydantic_messages_sdk.py | ||
| test_caching_redis_sdk.py | ||
| test_callback_logging_cache.py | ||
| test_databricks_json_schema_refs_sdk.py | ||
| test_device_code_login_guard_sdk.py | ||
| test_dual_cache_redis.py | ||
| test_generic_api_callback.py | ||
| test_google_genai_adapter_stream_usage_sdk.py | ||
| test_http2_wire.py | ||
| test_idle_upstream_reconnect_sdk.py | ||
| test_interception_converted_stream_options_sdk.py | ||
| test_langfuse_redaction_sdk.py | ||
| test_ocr_matrix_sdk.py | ||
| test_openai_sdk_client_sessions_wire.py | ||
| test_owned_redis_ports.py | ||
| test_provider_budget_redis.py | ||
| test_provider_error_and_extra_body_families_wire.py | ||
| test_redis_breaker_metrics_multiprocess_sdk.py | ||
| test_redis_cluster_iam_auth.py | ||
| test_redis_service_metrics.py | ||
| test_response_cache_sync_sdk.py | ||
| test_responses_bridge_stream_errors_sdk.py | ||
| test_responses_bridge_tool_search_sdk.py | ||
| test_router_behavior_wire.py | ||
| test_router_budget_limiter_wire.py | ||
| test_router_cooldown_fallback_sdk.py | ||
| test_router_include_fallback_errors_wire.py | ||
| test_router_lowest_latency_wire.py | ||
| test_router_redis_tls_url.py | ||
| test_router_sync_stream_fallback_wire.py | ||
| test_router_tool_call_stream_fallback_budget_wire.py | ||
| test_router_unsupported_params_fallback_wire.py | ||
| test_sagemaker_embedding_sdk.py | ||
| test_slack_daily_report_redis.py | ||
| test_stream_open_error_provider_headers_sdk.py | ||
| test_sync_stream_connection_release_wire.py | ||
| test_tool_loop_server_tool_replay_sdk.py | ||
| test_topaz_image_variation_trailing_slash_sdk.py | ||
| test_usage_based_routing_sdk_redis_reads.py | ||
| test_usage_routing_counter_ttl.py | ||
| test_vertex_batch_row_line_errors_sdk.py | ||
| test_vertex_gemini_image_batch_cost_sdk.py | ||
| test_vertex_gemini_stream_timeout_wire.py | ||
| test_vertex_partner_count_tokens_sdk.py | ||