litellm/tests/unit
mateo f20a6ae329 fix(responses): keep the bridged logprobs immutable and typed for the lint gates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-31 23:39:28 +00:00
..
a2a_protocol fix(a2a)!: bill completion-bridge agents at their own prices on /a2a send and stream (#49150) 2026-10-30 19:43:45 -07:00
accounting feat(accounting): add accounting contracts, PricingSnapshot and ledger schema (LIT-10193) (#48862) 2026-10-28 14:39:25 -07:00
anthropic_interface
batch_completion
batches feat(accounting): capture request usage through the durable meter (#49133) 2026-10-31 07:32:25 -07:00
caching test: deflake the json accumulator drain test and the armed routing prefetch test (#49505) 2026-10-31 15:14:33 -07:00
completion_extras fix(streaming): rebuild each choice of an n>1 chat stream on its own (#49002) 2026-10-29 18:29:14 -07:00
compression
containers refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
db_scripts
decisions fix(decisions)!: honor a deployment's own supports_audio_input and supports_video_input flags (#49207) 2026-10-31 01:09:46 -07:00
endpoints feat(vertex-ai)!: support gemini 3.1 flash tts (#31915) 2026-10-21 04:18:42 +00:00
enterprise refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
expected_fine_tuning_api
expected_responses_api_request
experimental_mcp_client fix(mcp): load deprecated package aliases on demand (#48615) 2026-10-27 19:46:19 +00:00
extraction feat(extraction): add /v1/extraction with Cloudflare Clairic support (#47473) 2026-10-20 15:51:45 -07:00
files fix(hosted_vllm): accept the vLLM production-stack router's file and batch objects (#49068) 2026-10-30 23:11:34 -07:00
fine_tuning fix(fine_tuning): stop forwarding api_version to the OpenAI client (#44995) 2026-10-21 01:58:25 +00:00
fixtures
gateway fix(proxy): keep the litellm command light so spawned workers answer uvicorn's first health ping (#45039) 2026-10-29 10:25:14 -07:00
google_genai fix(google_genai): read one-object and null spellings of repeated fields in countTokens and generateContent (#48771) 2026-10-28 21:46:06 -07:00
harness fix(streaming)!: send the include_usage chunk with an empty choices array (#47109) 2026-10-21 02:26:05 +00:00
images feat(dashscope): add image edits and text-to-speech (#48998) 2026-10-29 17:48:38 -07:00
integration_support
integrations fix(anthropic): answer 400 naming a non-string role or tool call id instead of 500 (#49237) 2026-10-31 21:13:24 +00:00
interactions fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
litellm_core_utils fix(streaming): preserve served models in OpenRouter streams (#49498) 2026-10-31 16:12:51 -07:00
litellm_proxy_extras feat(proxy): replace the pre-migration row check with a rolled-back dry run of the pending migrations (#48961) 2026-10-29 09:39:26 -07:00
llms fix(streaming): preserve served models in OpenRouter streams (#49498) 2026-10-31 16:12:51 -07:00
mcp_client refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
models
ocr
passthrough fix(vllm): send the pass-through key as a bearer on both /vllm routes (#46861) 2026-10-30 08:57:48 -07:00
proxy fix(proxy): serve /a2a/{agent_id}/message/stream routes (#49471) 2026-10-31 16:11:51 -07:00
rag
realtime_api feat(soniox): add realtime speech-to-text on /v1/realtime (#44822) 2026-10-29 07:58:38 -07:00
repositories refactor(proxy): inject gateway activity through a typed repository (#49116) 2026-10-30 16:25:11 -07:00
rerank_api
responses fix(responses): keep the bridged logprobs immutable and typed for the lint gates 2026-10-31 23:39:28 +00:00
router_strategy fix(router): restore four base behaviors the RoutingContext refactor changed (#49395) 2026-10-31 22:58:34 +00:00
router_utils fix(router): restore four base behaviors the RoutingContext refactor changed (#49395) 2026-10-31 22:58:34 +00:00
rust_bridge refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
sandbox fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
search
secret_managers feat(secret_managers): resolve oidc/google api_key references per request (#47703) 2026-10-29 08:44:46 -07:00
skills feat(skills): add a litellm-setup agent skill (#47731) 2026-10-28 15:44:43 -07:00
systemone fix(databricks): reuse base OAuth for decisions (#48395) (#48527) 2026-10-26 20:18:16 -07:00
telemetry
test_router fix(router): restore four base behaviors the RoutingContext refactor changed (#49395) 2026-10-31 22:58:34 +00:00
text_completions fix(proxy)!: bill streamed /v1/completions without stream_options like chat (#45366) 2026-10-29 17:18:51 -07:00
tracing
types fix(gemini): add thought_signature_in_tool_call_id to keep tool call ids short for clients that cap them (#47692) 2026-10-31 03:20:42 -07:00
vector_stores fix(exceptions)!: honor status code when mapping OpenAI invalid_request_error exceptions (#32537) 2026-10-29 08:05:47 -07:00
videos
__init__.py
AGENTS.md feat(vector_stores): add OpenSearch k-NN search as a vector store provider (#47759) 2026-10-29 08:15:05 -07:00
conftest.py fix(llms): move later developer messages for models declared system-first (#48897) 2026-10-29 04:12:13 -07:00
create_mock_standard_logging_payload.py
test_a2a_registry_lookup.py
test_acompletion_session_reuse_e2e.py
test_add_deployment_no_master_key.py
test_aembedding_session_reuse_e2e.py
test_anthropic_beta_gated_fields.py fix(anthropic)!: drop beta-gated body fields under drop_params (#48793) 2026-10-29 06:25:31 -07:00
test_anthropic_beta_headers_filtering.py fix(vertex_ai,azure_ai)!: auto-add the compaction, thinking display, and tool change betas on Claude routes (#44747) 2026-10-30 09:28:19 -07:00
test_anthropic_skills_transformation.py
test_assert_ci_coverage.py
test_assert_secret_absent.py ci: run the integration groups on GitHub Actions (#48985) 2026-10-29 12:50:08 -07:00
test_assert_workflow_dir_hygiene.py
test_audio_transcription_rust_bridge.py fix(bedrock): omit system prompt from audio transcription and report its token usage (#41565) 2026-10-28 19:23:15 -07:00
test_azure_ad_token_credential_resolution.py
test_azure_ai_grok_4_3_model_metadata.py
test_azure_ai_grok_4_6_model_metadata.py
test_baseten_glm_5_3_model_metadata.py
test_batch_completion_models_all_responses.py
test_bedrock_marengo_embed_3_model_metadata.py
test_build_core_distribution.py
test_cerebras_llama_model_metadata.py feat(models): add Cerebras Llama 3.1 hyphenated aliases (cerebras/llama-3.1-8b, cerebras/llama-3.1-70b) (#48157) (#48158) 2026-10-24 17:16:33 -07:00
test_chat_ui_responses_session.py
test_check_licenses.py
test_check_mcp_operation_boundary.py refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
test_check_migrations_lock_timeout.py
test_check_migrations_no_data_rewrites.py
test_check_openapi_docs.py
test_check_py310_typing_imports.py
test_check_test_quality.py
test_check_type_discipline.py
test_check_unbounded_in_lists.py
test_check_windows_wheel_install.py
test_circleci_checkout.py ci(circleci): clone the public repo over HTTPS so main jobs stop failing at checkout (#49394) 2026-10-31 02:53:54 -07:00
test_circleci_path_filter.py refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
test_circleci_run_migration_tests.py
test_circleci_rust_toolchain.py
test_claude_fable_5_config.py
test_claude_haiku_5_5_config.py
test_claude_opus_4_6_config.py
test_claude_opus_4_8_config.py
test_claude_opus_5_config.py
test_claude_sonnet_5_config.py
test_cloudflare_workers_ai_model_metadata.py
test_completion_timeout_resolution.py
test_component_entrypoint.py
test_compression.py
test_conftest_isolation.py refactor(router): extract PythonCore behind RouterCore and pass the router its config (#49119) 2026-10-31 12:54:23 -07:00
test_constants.py fix(router): cap the default OTPM reservation and reconcile token counters after each call (#49054) 2026-10-29 20:32:04 -07:00
test_container_router.py
test_cost_calculation_log_level.py
test_cost_calculator.py fix(cost): split a provider-reported total between input and output by the cost map rate ratio (#49357) 2026-10-31 09:04:50 -07:00
test_cost_map_guard.py fix(cost_map): mark fireworks_ai/minimax-m3 as vision and pin capabilities verified against live calls (#43390) 2026-10-29 00:23:11 -07:00
test_count_tokens_public_api.py
test_dashscope_image_generation.py feat(dashscope): add image edits and text-to-speech (#48998) 2026-10-29 17:48:38 -07:00
test_daybreak_model_metadata.py
test_deepseek_model_metadata.py
test_default_branch.py feat(lite): offer each model group's reasoning efforts in pi, OpenCode and Codex (#48949) 2026-10-29 18:15:44 -07:00
test_detect_changes.py
test_docker_no_network_on_deploy.py
test_dockerfile_apk_repository.py
test_dockerfile_bedrock_realtime_extra.py
test_dockerfile_non_root.py
test_drop_params_env_var.py
test_e2e_egress_sentinel.py
test_eager_tiktoken_load.py
test_elevenlabs_tts_pricing.py fix(cost-map): rolling registry audit, add elevenlabs api rates, together minimax-m3.1-flash, mistral voxtral 3 and azure_ai phi-4 dates (#47623) 2026-10-23 14:57:35 -07:00
test_empty_mock_response.py fix: honor empty mock completion responses (#48162) (#48266) 2026-10-28 00:56:45 +00:00
test_env_key_doc_gate.py
test_exception_exports.py
test_exception_header_preservation.py fix(openai): keep the upstream error body on e.response for OpenAI and Azure errors (#49079) 2026-10-29 23:30:02 -07:00
test_exception_mapping_request_attribute.py
test_fake_openai_endpoint.py
test_filter_out_litellm_params.py
test_fireworks_serverless_model_costs.py
test_gate_slot_lock.py
test_gemini_3_1_flash_lite_image_pricing.py
test_gemini_4_argon_model_metadata.py
test_gemini_tts_native_audio_pricing.py
test_get_blog_posts.py feat(proxy): add LITELLM_OFFLINE_MODE, one switch for the fetches LiteLLM makes on its own (#48096) 2026-10-23 16:54:19 -07:00
test_git_hooks.py
test_gpt_5_4_model_metadata.py
test_gpt_5_5_model_metadata.py
test_gpt_image_cost_calculator.py
test_gpt_realtime_mode.py
test_groq_streaming_encoding.py fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
test_guardrail_exception_status_codes.py
test_integration_run.py
test_internal_context.py feat(proxy)!: model family and access group entries in model_max_budget (#47722) 2026-10-28 15:40:57 -07:00
test_lazy_imports.py fix(health)!: openai realtime health check reports healthy with a missing or invalid api key (#41233) 2026-10-29 00:40:02 -07:00
test_lint_base_counts.py
test_lint_workflow_diff_gates.py fix(ci): measure PR gates from the target branch's current tip (#49078) 2026-10-29 18:28:16 -07:00
test_litellm_cli.py fix(proxy): keep the litellm command light so spawned workers answer uvicorn's first health ping (#45039) 2026-10-29 10:25:14 -07:00
test_litellm_overhead.py
test_litellm_params_reserved_keys.py
test_logging.py feat(proxy)!: add suppress_startup_output to turn off startup prints (#47708) 2026-10-29 03:47:04 -07:00
test_lowest_latency_zero_tokens.py
test_main.py fix(gemini): add thought_signature_in_tool_call_id to keep tool call ids short for clients that cap them (#47692) 2026-10-31 03:20:42 -07:00
test_main_module_header.py
test_mistral_large_4_model_metadata.py fix(cost-map): price Mistral Large 4 at its list price now that the launch sale ended (#48825) 2026-10-28 18:12:55 +00:00
test_mistral_medium_3_5_model_metadata.py
test_mistral_small_4_0_model_metadata.py
test_mistral_zai_glm_5_2_model_metadata.py
test_model_block_unblock.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_model_cost_aliases.py
test_model_param_helper.py
test_model_prices_and_context_window.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
test_model_response_normalization.py
test_muse_spark_1_1_model_metadata.py
test_muse_spark_1_2_model_metadata.py
test_muse_spark_1_3_model_metadata.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
test_mutation_report.py
test_nested_drop_params.py
test_no_top_level_test_invocations.py
test_non_chat_routes_open_llm_spans.py
test_openai_embedding_encoding_format_default.py
test_openai_record_replay_proxy.py
test_openai_service_tier_long_context_pricing.py
test_openapi_docs_gate.py fix(ci): build base native extension for OpenAPI lint (#49136) 2026-10-29 16:56:44 -07:00
test_package_layout.py
test_package_metadata.py
test_pre_commit_lint.py ci: run the dashboard API types and Terraform endpoint checks on every change that can alter the spec (#49022) 2026-10-29 07:19:06 -07:00
test_prisma_generate_if_needed.py
test_private_usage_aliases.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
test_process_helpers.py
test_project_alias_tracking.py
test_project_tags_pydantic.py
test_proxy_auth.py
test_pydantic_namespaces.py
test_rag_openai_ingestion.py
test_rate_limit_error_unification.py
test_read_rc_version.py
test_redact_string_in_error_paths.py
test_redis.py
test_redis_credential_provider.py
test_register_model_custom_pricing.py
test_register_model_zero_cost_persistence.py
test_replicate_model_key_format.py
test_repo_agent_skills.py feat(skills): add a litellm-setup agent skill (#47731) 2026-10-28 15:44:43 -07:00
test_responses_api_bridge_non_stream.py
test_responses_id_security.py fix(responses): resume background response streams with stream and starting_after (#48819) 2026-10-29 02:31:22 -07:00
test_responses_streaming_container_ownership.py
test_retrieve_batch_bedrock_dispatch.py
test_router_block_helpers.py
test_router_exception_redaction.py
test_router_get_settings.py
test_router_google_genai.py
test_router_model_cost_isolation.py fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166) 2026-10-29 08:06:12 -07:00
test_router_order_fallback.py
test_router_per_deployment_num_retries.py fix(images): honor max_retries and prevent Router retry amplification (#48705) 2026-10-28 10:46:47 -07:00
test_router_redis_init.py
test_router_retry_backoff_headers.py
test_router_retry_non_retryable_errors.py
test_router_retry_policy_update.py
test_router_silent_experiment.py refactor(router): extract PythonCore behind RouterCore and pass the router its config (#49119) 2026-10-31 12:54:23 -07:00
test_router_streaming_fallback_metadata.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
test_router_weighted_failover.py refactor(router): extract PythonCore behind RouterCore and pass the router its config (#49119) 2026-10-31 12:54:23 -07:00
test_ruff_strict_gate.py
test_sambanova_model_metadata.py
test_scaledown_model_prices.py
test_scheduler.py fix(scheduler): read shared queue from Redis so expired dead-replica entries stop blocking waiters (#49387) 2026-10-31 11:43:46 -07:00
test_secret_redaction.py refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
test_seed_tracing_fixtures.py
test_select_ui_test_scope.py fix(ci): measure PR gates from the target branch's current tip (#49078) 2026-10-29 18:28:16 -07:00
test_service_logger.py
test_setup_wizard.py
test_shared_session_integration.py
test_socket_policy.py
test_spend_logs_guard.py fix(ci): flag any schema.prisma change inside model LiteLLM_SpendLogs in the spend logs guard (LIT-10240) (#48905) 2026-10-28 15:52:15 -07:00
test_ssl_verify_unit.py
test_stream_chunk_builder_annotations.py fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166) 2026-10-29 08:06:12 -07:00
test_stream_chunk_builder_citations.py
test_stream_chunk_builder_images.py
test_streaming_connection_cleanup.py
test_system_message_format_bug.py
test_test_quality_gate.py
test_thinking_enabled.py
test_together_ai_model_metadata.py refactor(cost-map): drop the legacy max_tokens field in favor of max_input_tokens and max_output_tokens (#48471) 2026-10-26 17:03:05 -07:00
test_tool_loop.py fix(responses): replay the original Responses item order, phase, and boundaries on the next chat turn (#48912) 2026-10-29 00:56:02 -07:00
test_type_check_gate.py
test_type_discipline_gate.py
test_typesafe_model_metadata.py
test_unit_shard_missing_paths.py
test_unit_shard_per_test_timeout.py
test_utils.py fix(openrouter)!: list OpenRouter's live catalog for openrouter/* when check_provider_endpoint is on (#48820) 2026-10-30 12:19:51 -07:00
test_utils_get_model_info.py fix(bedrock): price claude_platform calls from their own cost map rows (#48150) 2026-10-24 13:54:18 -07:00
test_utils_get_optional_params.py fix(openai): warn when the GPT-5 reasoning effort gate drops a level under drop_params (#47761) 2026-10-29 04:46:29 -07:00
test_utils_module_docstring.py
test_uuid_helper.py fix: support Python 3.15 (#48359) 2026-10-26 16:00:11 +00:00
test_vcr_classification.py
test_vcr_conftest_common_banner.py
test_vcr_filters.py
test_vcr_leak_guard.py
test_vcr_redis_persister.py
test_vcr_safe_body_matcher.py
test_version.py
test_vertex_ai_xai_grok_prompt_caching_metadata.py
test_video_generation.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_with_dashboard_node.py
test_ws_vcr.py
test_xai_grok_4_3_model_metadata.py
test_xai_responses_auto_routing.py