litellm/tests/test_litellm
yassin 15ff8d18e7 test(proxy): cover the CLI single-model branch of /model/info JSON serialization
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 10:07:10 +00:00
..
a2a_protocol feat(agentcore-a2a): derive runtime session id from A2A message.contextId (#39371) 2026-09-02 12:40:24 -07:00
anthropic_interface refactor(rust): remove per-request enablement arguments (#39928) 2026-09-07 10:43:45 -07:00
batches Merge pull request #39626 from BerriAI/litellm_batch_ui_logs 2026-09-08 15:19:25 -07:00
caching Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_redis_pool_timeout_counts_as_timeout 2026-09-12 15:26:08 -07:00
completion_extras fix(responses bridge): keep mid-conversation system messages in input 2026-09-08 11:44:49 -07:00
compression
containers fix(containers): page upstream until a non-admin container list fills its limit 2026-09-02 21:26:34 -07:00
endpoints fix(speech): honor pcm response_format for Gemini TTS and reject unsupported containers 2026-08-29 16:36:34 -07:00
enterprise Merge pull request #39836 from BerriAI/litellm_lit_6975_bedrock_files_delete_list 2026-09-12 21:13:27 -07:00
expected_fine_tuning_api
expected_responses_api_request
experimental_mcp_client fix(mcp): bind discovery caching to credentials and active callers 2026-09-11 14:49:00 -07:00
fixtures/together_ai_sync
google_genai
images
integrations Merge pull request #40772 from BerriAI/litellm_lit6982_bucket_config_includes 2026-09-12 15:21:56 -07:00
interactions
litellm_core_utils Merge pull request #39846 from BerriAI/litellm_bedrock_mantle_govcloud_cost_row 2026-09-12 21:13:58 -07:00
llms Merge pull request #39846 from BerriAI/litellm_bedrock_mantle_govcloud_cost_row 2026-09-12 21:13:58 -07:00
models test(models): stop the password serialization test matching field-name substrings 2026-09-14 09:59:45 +02:00
ocr Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ocr_custom_pricing 2026-09-12 15:42:59 -07:00
passthrough fix(passthrough): parse Bedrock stream spend incrementally instead of buffering the whole response (#40724) 2026-09-11 09:53:37 -07:00
proxy test(proxy): cover the CLI single-model branch of /model/info JSON serialization 2026-09-14 10:07:10 +00:00
rag fix(rag): let the managed store's params win over caller kwargs on the search call 2026-09-02 18:59:28 -07:00
realtime_api refactor(realtime): move Meta Muse Voice onto BaseRealtimeConfig 2026-09-11 20:12:15 -07:00
repositories feat(auto-router): show routed model and savings in Claude Code and Codex (#40330) 2026-09-11 12:52:42 -07:00
rerank_api chore: merge litellm_internal_staging into litellm_fix_dashscope_rerank_endpoint 2026-09-02 15:07:02 -07:00
responses fix(responses): carry the reasoning summary as an alias, not inside reasoning_effort 2026-09-12 17:49:30 -07:00
router_strategy fix(router): fall back from unhealthy auto-router tier (#40757) 2026-09-11 11:51:55 -07:00
router_utils feat(proxy): expose complexity routing headers (#40792) 2026-09-11 17:14:40 -07:00
rust_bridge refactor(ocr): complete native lifecycle and preserve Azure auth (#40734) 2026-09-12 11:56:49 -07:00
sandbox
secret_managers feat(secret_managers): support customer-managed KMS key for virtual keys stored in AWS Secrets Manager (#40475) 2026-09-11 09:54:08 -07:00
skills feat(skills): semantic search over the LiteLLM-hosted skill registry (#39401) 2026-09-07 12:28:38 -07:00
test_router fix(router): count deployment itpm reservations off the event loop 2026-09-07 21:29:26 -07:00
types feat(bedrock): thread aws_session_tags into STS AssumeRole 2026-09-09 12:37:36 -07:00
vector_stores Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_python_version_ci 2026-09-03 00:16:19 -07:00
videos
__init__.py
conftest.py fix(image_edit): await an async transform hook and hide the URL policy verdict from callers 2026-09-04 23:20:21 -07:00
log.txt
readme.md
test_a2a_registry_lookup.py
test_acompletion_session_reuse_e2e.py
test_add_deployment_no_master_key.py
test_aembedding_session_reuse_e2e.py
test_anthropic_beta_headers_filtering.py
test_anthropic_skills_transformation.py
test_anthropic_sonnet_1hr_cache_pricing.py
test_assert_ci_coverage.py
test_assert_workflow_dir_hygiene.py
test_audio_transcription_rust_bridge.py refactor(rust): remove per-request enablement arguments (#39928) 2026-09-07 10:43:45 -07:00
test_auto_update_price_and_context_window_file.py feat(cost_map): derive source_revision from the loaded bytes instead of a _metadata stamp 2026-09-07 17:47:51 -07:00
test_azure_ad_token_credential_resolution.py
test_azure_ai_grok_4_3_model_metadata.py
test_azure_ai_grok_4_6_model_metadata.py feat(azure_ai): add grok-4.6 to the model cost map 2026-09-02 16:03:45 -07:00
test_azure_audio_price_aliases.py
test_baseten_glm_5_3_model_metadata.py fix(registry): mark baseten GLM-5.3 as vision-capable per Baseten vision docs 2026-09-04 20:10:58 +00:00
test_batch_completion_models_all_responses.py
test_bedrock_anthropic_1hr_cache_pricing.py
test_bedrock_batch_pricing.py
test_bedrock_marengo_embed_3_model_metadata.py fix(bedrock): bill Marengo embeddings per request instead of per estimated token 2026-09-07 18:20:28 -07:00
test_bedrock_usgov_pricing.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_budget_ratchet_check.py
test_chat_ui_responses_session.py
test_check_licenses.py
test_check_migrations_no_data_rewrites.py
test_check_py310_typing_imports.py fix: keep litellm importable on Python 3.10 and guard 3.11-only typing imports in CI (#39448) 2026-09-02 18:27:19 -07:00
test_check_test_quality.py
test_check_type_discipline.py
test_circleci_path_filter.py
test_circleci_rust_toolchain.py fix(ci): pin workflow toolchain dependencies 2026-09-02 12:16:25 -07:00
test_claude_fable_5_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_claude_haiku_4_5_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_claude_opus_4_6_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_claude_opus_4_8_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_claude_opus_5_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_claude_sonnet_4_6_config.py
test_claude_sonnet_5_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_cloudflare_workers_ai_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_command_r7b_pricing.py
test_completion_timeout_resolution.py
test_component_entrypoint.py feat(proxy): share database connections across workers with an in-container pgbouncer (#39683) 2026-09-10 22:14:37 +00:00
test_compression.py
test_conftest.py
test_conftest_isolation.py
test_constants.py
test_container_router.py
test_cost_calculation_log_level.py
test_cost_calculator.py Merge pull request #39846 from BerriAI/litellm_bedrock_mantle_govcloud_cost_row 2026-09-12 21:13:58 -07:00
test_cost_map_guard.py feat(cost_map): derive source_revision from the loaded bytes instead of a _metadata stamp 2026-09-07 17:47:51 -07:00
test_count_tokens_public_api.py
test_dashscope_image_generation.py
test_daybreak_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_deepseek_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_default_branch.py ci: avoid duplicate default branch fetches 2026-09-07 15:28:01 -07:00
test_detect_changes.py
test_dockerfile_apk_repository.py fix(docker): add public Wolfi apk repo to runtime image (#39033) 2026-09-01 15:11:15 -07:00
test_dockerfile_bedrock_realtime_extra.py fix(docker): install bedrock-realtime extra in monolith proxy images (#39223) 2026-09-01 17:51:55 -07:00
test_dockerfile_non_root.py
test_drop_params_env_var.py fix(init): keep non-flag LITELLM_DROP_PARAMS values on with a warning 2026-09-07 22:13:23 -07:00
test_e2e_egress_sentinel.py
test_eager_tiktoken_load.py
test_env_key_doc_gate.py
test_exception_exports.py
test_exception_header_preservation.py fix(bedrock): keep x-amzn-RequestId on chat error responses (#40089) 2026-09-07 17:16:47 -07:00
test_exception_mapping_request_attribute.py
test_filter_out_litellm_params.py
test_fireworks_serverless_model_costs.py test(model_prices): use local cost map for Fireworks cost coverage 2026-09-12 01:00:11 +00:00
test_friendli_glm_5_3_flash_model_metadata.py fix(friendli): declare GLM-5.3-Flash reasoning efforts as explicit levels 2026-08-31 13:25:51 -07:00
test_friendli_glm_5_3_model_metadata.py fix(friendli): track GLM-5.3 discounted live pricing and declare effort levels 2026-08-31 13:26:37 -07:00
test_gate_slot_lock.py ci: avoid duplicate default branch fetches 2026-09-07 15:28:01 -07:00
test_gemini_3_1_flash_lite_image_pricing.py
test_gemini_tts_native_audio_pricing.py
test_get_blog_posts.py
test_git_hooks.py
test_github_close_low_quality_prs.py
test_github_review_gate.py
test_github_triage_with_llm.py
test_github_triage_workflows.py
test_gpt_5_4_model_metadata.py
test_gpt_5_5_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_gpt_image_cost_calculator.py fix(registry): drop gpt-image-2 text output price, add openrouter minimax-m3 and qwen3.7-plus 2026-09-04 15:03:29 +00:00
test_gpt_realtime_mode.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_groq_streaming_encoding.py
test_guardrail_exception_status_codes.py
test_lazy_imports.py Revert "perf: lazy-load SDK symbols so import litellm stays under 60 MB RSS (…" 2026-09-05 16:07:09 -07:00
test_lint_workflow_diff_gates.py ci(lint): gate top-level tests/e2e and litellm files in the diff-scoped lint steps 2026-09-01 22:30:46 -07:00
test_litellm_params_reserved_keys.py
test_logging.py fix(logging): redact credential query params from the uvicorn access log (#39293) 2026-09-02 15:10:36 -07:00
test_lowest_latency_zero_tokens.py
test_main.py Merge pull request #40771 from BerriAI/litellm_regression_coverage_followup 2026-09-12 15:29:36 -07:00
test_main_module_header.py
test_mistral_medium_3_5_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_mistral_small_4_0_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_mistral_zai_glm_5_2_model_metadata.py
test_model_block_unblock.py
test_model_cost_aliases.py
test_model_param_helper.py
test_model_prices_schema.py fix(cost-map): keep minimal withheld on Bedrock gpt-5.4 and gpt-5.5 2026-09-11 13:46:28 -07:00
test_model_response_normalization.py
test_muse_spark_1_1_model_metadata.py
test_muse_spark_1_2_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_muse_spark_1_3_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_mutation_report.py
test_nested_drop_params.py
test_non_chat_routes_open_llm_spans.py
test_openai_embedding_encoding_format_default.py test(embeddings): move legacy intercepts to the wire for the omitted-format path 2026-08-29 12:04:44 -07:00
test_openai_service_tier_long_context_pricing.py fix(tests): fold the local price map into the provider model sets 2026-09-03 13:09:46 -07:00
test_pre_commit_lint.py ci: follow the default branch in development tooling 2026-09-07 14:34:45 -07:00
test_prisma_generate_if_needed.py
test_project_alias_tracking.py
test_project_tags_pydantic.py
test_proxy_auth.py
test_rag_openai_ingestion.py
test_rate_limit_error_unification.py
test_redact_string_in_error_paths.py fix(proxy): stop leaking internal exception details to clients (#39380) 2026-09-02 17:32:00 -07:00
test_redis.py test(redis): pin ElastiCache IAM signing and TLS coercion invariants 2026-09-10 10:13:03 -04:00
test_redis_credential_provider.py test(redis): pin ElastiCache IAM signing and TLS coercion invariants 2026-09-10 10:13:03 -04:00
test_register_model_custom_pricing.py fix(cost): bill off-peak rates for deployments that set only off_peak_pricing 2026-09-01 12:14:46 -07:00
test_register_model_zero_cost_persistence.py
test_replicate_model_key_format.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_responses_api_bridge_non_stream.py
test_responses_id_security.py fix(proxy): authorize every Responses API id, not only the ones the proxy issued (#39548) 2026-09-11 11:47:05 -07:00
test_responses_streaming_container_ownership.py
test_retrieve_batch_bedrock_dispatch.py
test_router.py fix(router): preserve Azure Entra ID params in reusable credentials (#40889) 2026-09-12 21:13:45 -07:00
test_router_block_helpers.py
test_router_exception_redaction.py
test_router_google_genai.py
test_router_model_cost_isolation.py perf(proxy): reuse cached model group and deployment info in budget reservation (#40593) 2026-09-10 11:22:46 -07:00
test_router_order_fallback.py fix(router): skip the refusing deployment when retrying a non-transient error 2026-09-05 22:25:13 -07:00
test_router_per_deployment_num_retries.py refactor(router): resolve retry policy by exception MRO and add DefaultRetries 2026-09-04 16:09:01 -07:00
test_router_redis_init.py
test_router_retry_backoff_headers.py
test_router_retry_non_retryable_errors.py
test_router_retry_policy_update.py fix: preserve shared optional callbacks 2026-09-02 03:28:04 +00:00
test_router_silent_experiment.py
test_router_streaming_fallback_metadata.py
test_router_weighted_failover.py fix(router): skip the refusing deployment when retrying a non-transient error 2026-09-05 22:25:13 -07:00
test_ruff_strict_gate.py fix: address cross-version CI failures 2026-09-02 14:17:19 -07:00
test_sambanova_model_metadata.py
test_secret_redaction.py fix(proxy): stop leaking internal exception details to clients (#39380) 2026-09-02 17:32:00 -07:00
test_select_ui_test_scope.py test(ui): align workflow expectations with Vitest 4 2026-09-08 16:13:58 -07:00
test_service_logger.py
test_setup_wizard.py
test_shared_session_integration.py
test_ssl_verify_unit.py
test_stream_chunk_builder_annotations.py
test_stream_chunk_builder_citations.py
test_stream_chunk_builder_images.py
test_streaming_connection_cleanup.py
test_sync_together_ai_models.py feat(cost_map): derive source_revision from the loaded bytes instead of a _metadata stamp 2026-09-07 17:47:51 -07:00
test_system_message_format_bug.py
test_test_quality_gate.py Merge pull request #39870 from BerriAI/litellm_lit_6917_make_check_test_tree_ruff 2026-09-07 10:55:28 -07:00
test_thinking_enabled.py
test_together_ai_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_type_check_gate.py
test_type_discipline_gate.py
test_utils.py Merge pull request #40929 from BerriAI/litellm_fireworks_short_key_lookup 2026-09-12 20:49:44 -07:00
test_utils_module_docstring.py
test_uuid_helper.py
test_vcr_safe_body_matcher.py
test_video_generation.py fix(cost): carry output_cost_per_second_720p through model info 2026-09-09 14:09:25 +00:00
test_with_dashboard_node.py
test_xai_grok_4_3_model_metadata.py
test_xai_responses_auto_routing.py

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py