scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-25 01:02:15 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 131
litellm/tests/test_litellm
History
yuneng d47008129c test(llms): migrate phase 7 provider unit tests to tests/unit 
Move the wave 1 phase 7 batch (fireworks_ai, gemini, gigachat, github_copilot; 20 files) from tests/test_litellm to tests/unit after judging every test function under a behaviour mutation. Seven wiring or mock-echo tests that stayed green are deleted. The fireworks cost calculator tests get a local model_cost save/restore fixture since the tests/unit tree has no shared conftest for it

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 08:08:12 +00:00
..
a2a_protocol fix(a2a): keep the upstream status on card discovery failures and inject the card client in tests 2026-09-16 23:55:57 +00:00
anthropic_interface/exceptions refactor(rust_bridge): give chat completions, messages and responses the ocr dispatch shape 2026-09-16 15:02:12 -07:00
batches fix(cost): keep the deployment's OCR page rate when the model has no published price 2026-09-19 01:59:39 -07:00
caching fix(budgets): page end-user cache invalidation after a budget reset 2026-09-16 12:12:18 -07:00
chat_completions route stuff through dispatch no direct main 2026-09-17 11:06:46 -07:00
completion_extras fix(responses_bridge): forward verbosity as text.verbosity 2026-09-16 23:32:32 +00:00
compression merge: main into litellm_headroom_protect_cached_prefix 2026-09-15 04:12:28 +00:00
containers test: delete assertions that pin vendor cost map facts 2026-09-18 00:28:49 +00:00
endpoints
enterprise Merge pull request #41942 from BerriAI/litellm_vllm_batch_runner 2026-09-19 15:44:26 -07:00
expected_fine_tuning_api
expected_responses_api_request
experimental_mcp_client test(mcp): assert redirect rejection without SDK wording 2026-09-19 16:01:42 -07:00
fixtures/together_ai_sync
google_genai fix(google_genai): drop non-object tool parameters instead of forwarding them 2026-09-19 19:19:33 -07:00
images
integrations Merge pull request #41740 from BerriAI/litellm_otel_v2_langfuse_llm_spans_only 2026-09-19 17:24:50 -07:00
interactions refactor(interactions): remove expired use_legacy_interactions_schema shim 2026-09-17 20:14:07 +00:00
litellm_core_utils merge: origin/main into litellm_lit_7346_multi_choice_stream_guardrails 2026-09-19 17:34:43 -07:00
llms test(llms): migrate phase 7 provider unit tests to tests/unit 2026-09-20 08:08:12 +00:00
messages route stuff through dispatch no direct main 2026-09-17 11:06:46 -07:00
models fix(proxy): estimate auto-router baseline costs from durable cache history 2026-09-19 12:44:47 -07:00
ocr feat(rust): add Amazon Textract to litellm.ocr and sign provider requests after host hooks 2026-09-19 09:11:52 -07:00
passthrough test(passthrough): expect absent query params 2026-09-16 17:58:50 +00:00
proxy Merge pull request #42071 from BerriAI/litellm_remove_dead_telemetry_flag 2026-09-19 21:48:02 -07:00
rag fix(s3_vectors): embed registered-store ingests with the store's embedding model 2026-09-19 04:31:15 -07:00
realtime_api feat(vertex_ai): stream Chirp speech-to-text over /v1/realtime 2026-09-17 17:57:40 -07:00
repositories fix(proxy): estimate auto-router baseline costs from durable cache history 2026-09-19 12:44:47 -07:00
rerank_api ci(unit): fail a hung test in 120s with a traceback instead of idling the shard to its step timeout 2026-09-19 02:53:36 -07:00
responses Merge pull request #41953 from BerriAI/litellm_bridge_drop_tool_search 2026-09-19 04:42:28 -07:00
router_strategy Merge pull request #41617 from BerriAI/litellm_fuse_model_profile_presets 2026-09-19 16:17:19 -07:00
router_utils Merge pull request #42045 from BerriAI/litellm_lit_8201_notfound_retry_policy 2026-09-19 19:37:45 -07:00
rust_bridge refactor(rust): rename legacy callback adapter crate 2026-09-19 23:06:26 +00:00
sandbox
secret_managers Merge remote-tracking branch 'origin/main' into litellm_vault_login_secret_namespace 2026-09-18 08:57:50 +00:00
skills
test_router test(router): drop docstrings from the shared tpm regression tests 2026-09-19 01:06:50 +00:00
types fix(bedrock): merge main and thread aws_session_tags through the auth struct 2026-09-18 10:41:52 -07:00
vector_stores fix(bedrock): forward userContext in Knowledge Base Retrieve requests 2026-09-16 11:50:36 -07:00
videos
__init__.py
conftest.py fix(bedrock): gate Invoke tool search on the model map's supports_tool_search flag 2026-09-16 18:23:14 -07:00
log.txt
readme.md
test_a2a_registry_lookup.py fix(a2a): Entra credentials own the chat route bearer over a stored api_key or authorization header 2026-09-17 16:09:36 -07:00
test_acompletion_session_reuse_e2e.py
test_add_deployment_no_master_key.py
test_aembedding_session_reuse_e2e.py
test_anthropic_beta_headers_filtering.py fix(anthropic): register thinking-binding-controls-2026-08-01 in beta headers config 2026-09-15 15:34:24 +08:00
test_anthropic_skills_transformation.py
test_assert_ci_coverage.py test: add CircleCI integration contract foundation 2026-09-14 03:30:52 -07:00
test_assert_workflow_dir_hygiene.py
test_audio_transcription_rust_bridge.py refactor(rust_bridge): group route modules into packages and split ocr into main and rust 2026-09-16 20:34:51 +00:00
test_auto_update_price_and_context_window_file.py
test_azure_ad_token_credential_resolution.py
test_azure_ai_grok_4_3_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_azure_ai_grok_4_6_model_metadata.py test: keep behavior tests that read the cost map for a later fixture rewrite 2026-09-18 04:52:06 +00:00
test_baseten_glm_5_3_model_metadata.py test: drop tests that pin provider-owned cost map values 2026-09-18 03:55:51 +00:00
test_batch_completion_models_all_responses.py
test_bedrock_marengo_embed_3_model_metadata.py test: drop tests that pin provider-owned cost map values 2026-09-18 03:55:51 +00:00
test_budget_ratchet_check.py
test_chat_ui_responses_session.py
test_check_licenses.py
test_check_migrations_no_data_rewrites.py fix(migrations-check): read the table name past comments, ignore referential SET DEFAULT 2026-09-16 18:14:36 +00:00
test_check_py310_typing_imports.py
test_check_test_quality.py ci(tests): wire tests/unit into CircleCI and drain legacy unit shards green 2026-09-20 07:05:42 +00:00
test_check_type_discipline.py
test_circleci_path_filter.py fix(mcp): preserve session expiry signals and scope dependency CI 2026-09-18 22:52:10 -07:00
test_circleci_rust_toolchain.py
test_claude_fable_5_config.py test: keep the pinning-test removal free of unrelated reformatting 2026-09-18 04:27:28 +00:00
test_claude_opus_4_6_config.py test: keep the pinning-test removal free of unrelated reformatting 2026-09-18 04:27:28 +00:00
test_claude_opus_4_8_config.py test: keep the pinning-test removal free of unrelated reformatting 2026-09-18 04:27:28 +00:00
test_claude_opus_5_config.py test: keep the pinning-test removal free of unrelated reformatting 2026-09-18 04:27:28 +00:00
test_claude_sonnet_5_config.py test: keep the pinning-test removal free of unrelated reformatting 2026-09-18 04:27:28 +00:00
test_cloudflare_workers_ai_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_completion_timeout_resolution.py
test_component_entrypoint.py
test_compression.py
test_conftest.py
test_conftest_isolation.py
test_constants.py
test_container_router.py
test_cost_calculation_log_level.py
test_cost_calculator.py test(cost): point dated snapshot tests at a date the cost map cannot carry 2026-09-19 08:00:12 -07:00
test_cost_map_guard.py
test_count_tokens_public_api.py
test_dashscope_image_generation.py test: keep the pinning-test removal free of unrelated reformatting 2026-09-18 04:27:28 +00:00
test_daybreak_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_deepseek_model_metadata.py test: drop tests that pin provider-owned cost map values 2026-09-18 03:55:51 +00:00
test_default_branch.py
test_detect_changes.py
test_dockerfile_apk_repository.py
test_dockerfile_bedrock_realtime_extra.py fix(bedrock): keep realtime SDK error range inside websocket close reason 2026-09-17 01:57:33 +00:00
test_dockerfile_non_root.py
test_drop_params_env_var.py
test_e2e_egress_sentinel.py
test_eager_tiktoken_load.py
test_env_key_doc_gate.py
test_exception_exports.py
test_exception_header_preservation.py
test_exception_mapping_request_attribute.py
test_filter_out_litellm_params.py
test_fireworks_serverless_model_costs.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_gate_slot_lock.py
test_gemini_3_1_flash_lite_image_pricing.py test: drop gemini-3.1-flash-lite-image capability pins 2026-09-16 19:21:50 +00:00
test_gemini_tts_native_audio_pricing.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_get_blog_posts.py
test_git_hooks.py
test_gpt_5_4_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_gpt_5_5_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_gpt_image_cost_calculator.py test: drop remaining tests that pin cost-map vendor facts 2026-09-15 21:34:00 +00:00
test_gpt_realtime_mode.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_groq_streaming_encoding.py
test_guardrail_exception_status_codes.py
test_lazy_imports.py perf: defer fastapi and tiktoken BPE imports out of import litellm 2026-09-17 08:59:48 +00:00
test_lint_workflow_diff_gates.py
test_litellm_params_reserved_keys.py
test_logging.py fix(logging): stamp scrubbed records with a private sentinel a caller cannot supply 2026-09-16 17:30:49 -07:00
test_lowest_latency_zero_tokens.py
test_main.py merge: origin/main into litellm_lit_7346_multi_choice_stream_guardrails 2026-09-19 17:34:43 -07:00
test_main_module_header.py
test_mistral_medium_3_5_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_mistral_small_4_0_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_mistral_zai_glm_5_2_model_metadata.py test: keep behavior tests that read the cost map for a later fixture rewrite 2026-09-18 04:52:06 +00:00
test_model_block_unblock.py
test_model_cost_aliases.py
test_model_param_helper.py
test_model_prices_schema.py fix(cost): bill DeepSeek V4.1 Flash and V4 Pro at their off-peak rates outside peak hours 2026-09-19 04:31:55 -07:00
test_model_response_normalization.py
test_muse_spark_1_1_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_muse_spark_1_2_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_muse_spark_1_3_model_metadata.py test: delete assertions that pin vendor cost map facts 2026-09-18 00:28:49 +00:00
test_mutation_report.py
test_nested_drop_params.py
test_non_chat_routes_open_llm_spans.py
test_openai_embedding_encoding_format_default.py
test_openai_service_tier_long_context_pricing.py test: delete assertions that pin vendor cost map facts 2026-09-18 00:28:49 +00:00
test_pre_commit_lint.py
test_prisma_generate_if_needed.py
test_project_alias_tracking.py
test_project_tags_pydantic.py
test_proxy_auth.py
test_rag_openai_ingestion.py
test_rate_limit_error_unification.py fix(proxy): remove duplicate user budget hook that 429'd zero-cost models 2026-09-16 15:42:15 -07:00
test_redact_string_in_error_paths.py
test_redis.py
test_redis_credential_provider.py
test_register_model_custom_pricing.py
test_register_model_zero_cost_persistence.py
test_replicate_model_key_format.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_responses_api_bridge_non_stream.py
test_responses_id_security.py
test_responses_streaming_container_ownership.py
test_retrieve_batch_bedrock_dispatch.py
test_router.py Merge pull request #42045 from BerriAI/litellm_lit_8201_notfound_retry_policy 2026-09-19 19:37:45 -07:00
test_router_block_helpers.py
test_router_exception_redaction.py
test_router_google_genai.py
test_router_model_cost_isolation.py fix(router): preserve discovered limits and model info fallbacks 2026-09-17 00:23:50 +00:00
test_router_order_fallback.py
test_router_per_deployment_num_retries.py
test_router_redis_init.py
test_router_retry_backoff_headers.py
test_router_retry_non_retryable_errors.py fix(timing): union provider timing windows and anchor detailed pre-processing at receive time 2026-09-19 00:48:17 +00:00
test_router_retry_policy_update.py fix(proxy): persist only the router settings keys the request set 2026-09-18 14:01:05 -07:00
test_router_silent_experiment.py fix(router): snapshot shadow kwargs per target so concurrent shadows never share metadata 2026-09-16 04:31:43 +00:00
test_router_streaming_fallback_metadata.py
test_router_weighted_failover.py
test_ruff_strict_gate.py
test_sambanova_model_metadata.py test: keep behavior tests that read the cost map for a later fixture rewrite 2026-09-18 04:52:06 +00:00
test_secret_redaction.py fix(logging): scan each log record once and collapse base64 payloads before the secret regex 2026-09-12 19:17:26 -07:00
test_select_ui_test_scope.py
test_service_logger.py
test_setup_wizard.py
test_shared_session_integration.py
test_ssl_verify_unit.py
test_stream_chunk_builder_annotations.py
test_stream_chunk_builder_citations.py
test_stream_chunk_builder_images.py
test_streaming_connection_cleanup.py
test_sync_together_ai_models.py
test_system_message_format_bug.py
test_test_quality_gate.py ci(tests): wire tests/unit into CircleCI and drain legacy unit shards green 2026-09-20 07:05:42 +00:00
test_thinking_enabled.py
test_together_ai_model_metadata.py test: delete assertions that pin vendor cost map facts 2026-09-18 00:28:49 +00:00
test_type_check_gate.py
test_type_discipline_gate.py
test_typesafe_model_metadata.py feat(proxy): add TypeSafe Jev passthrough spend tracking 2026-09-17 15:53:19 +00:00
test_unit_shard_per_test_timeout.py test(ci): drop the structure-only assertion on the shard script; the parametrized hang test covers both invocations 2026-09-19 03:20:48 -07:00
test_utils.py fix(proxy): estimate auto-router baseline costs from durable cache history 2026-09-19 12:44:47 -07:00
test_utils_module_docstring.py
test_uuid_helper.py
test_vcr_safe_body_matcher.py
test_vertex_ai_xai_grok_prompt_caching_metadata.py test: keep behavior tests that read the cost map for a later fixture rewrite 2026-09-18 04:52:06 +00:00
test_video_generation.py test: delete assertions that pin vendor cost map facts 2026-09-18 00:28:49 +00:00
test_with_dashboard_node.py
test_xai_grok_4_3_model_metadata.py test: keep tests that survive correct cost-map updates 2026-09-15 22:16:08 +00:00
test_xai_responses_auto_routing.py fix(xai): keep 'instructions' on the xAI Responses API so system messages survive web_search bridging 2026-09-16 01:35:13 +00:00

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py