scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-13 23:11:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 101
litellm/tests/test_litellm
History
Exact
Yucheng Zhu 29ef0504d7 test(rate-limits): add direct litellm_metadata lookup coverage 
The new _stash_value_in_internal_metadata helper routes stash writes to the
proxy-internal metadata bucket, which can be litellm_metadata on Responses
routes. The lookup logic was updated to read from both channels, but the test
only verified the legacy metadata path.

Add explicit assertion testing lookup when values live in
litellm_params["litellm_metadata"] to close the coverage gap Greptile flagged.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-07-30 10:21:34 -07:00
..
a2a_protocol fix(responses): stop scheduling sync success_handler concurrently with async_success_handler (#32239) 2026-07-07 09:13:50 -07:00
anthropic_interface fix(rust): route agentic-completion-hook /messages requests to Python for all stream modes (#34126) 2026-07-22 00:36:47 +00:00
batches Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
caching Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/test-coverage-mutation-analysis-e42223 2026-07-23 23:52:11 -07:00
completion_extras chore: merge litellm_internal_staging into litellm_fix_responses_bridge_streaming_contract 2026-07-25 00:20:31 +00:00
containers fix(containers): record ownership for service-account keys + fix Prisma Json serialization (#28990) 2026-05-27 21:00:07 -07:00
enterprise test(managed-files): call store_unified_file_id twice and assert upsert payloads 2026-07-29 18:51:25 -07:00
expected_fine_tuning_api
expected_responses_api_request test(responses): replace perma-skip azure shell e2e with offline coverage (#32444) 2026-07-08 10:01:41 -07:00
experimental_mcp_client fix(mcp): sanitize Anthropic tool schemas and stop encoding gateway names 2026-07-17 10:33:28 -07:00
google_genai chore: litellm oss staging 250626 (#31305) 2026-06-25 21:00:28 -07:00
images fix(image_edit): read vertex_project/location from litellm_params in Imagen get_complete_url 2026-04-21 15:03:40 +05:30
integrations fix(otel): make OTLP export work against Grafana Cloud (#35060) 2026-07-29 13:43:33 -07:00
interactions fix(tests): assert Content variants are identified by type, not by the discriminator keyword (#35161) 2026-07-29 17:49:30 -07:00
litellm_core_utils Merge pull request #34675 from BerriAI/litellm_tool_spend_rollup 2026-07-27 15:31:19 -07:00
llms Merge pull request #35188 from BerriAI/litellm_remove_dead_bedrock_invoke_path 2026-07-29 21:21:25 -07:00
models feat(litellm): add models and repository layers (#29686) 2026-06-06 20:59:33 -07:00
ocr test: litellm fix failing tests (#32577) 2026-07-09 13:54:45 -07:00
passthrough fix(passthrough): stream non-sse passthrough responses instead of buffering in memory (#32386) 2026-07-07 20:51:15 -07:00
proxy test(rate-limits): add direct litellm_metadata lookup coverage 2026-07-30 10:21:34 -07:00
rag fix(rag): track LLM completion usage and spend for /v1/rag/query (#32438) 2026-07-17 17:45:27 +00:00
realtime_api fix(realtime): preserve nested transcription model and session-first model priority 2026-07-03 14:21:22 -07:00
repositories fix(team): make team member add atomic to prevent concurrent-add member loss (#34185) 2026-07-22 21:47:22 +00:00
rerank_api fix(rerank): log optional_rerank_params at debug to stop leaking request content (#32533) 2026-07-08 18:48:03 -07:00
responses fix(responses): strip include_usage from stream_options instead of dropping the param 2026-07-24 16:46:49 -07:00
router_strategy fix(complexity_router): propagate turn_off_message_logging to internal sub-calls 2026-07-29 18:28:48 -07:00
router_utils fix(proxy): reject model writes that corrupt an auto-router pseudo-model 2026-07-28 20:25:07 -07:00
sandbox feat: add opensandbox sandbox provider (#31024) 2026-06-23 09:05:13 -07:00
secret_managers fix(bedrock): stop replaying expired Google OIDC tokens to STS on guardrail auth (#34637) 2026-07-25 16:27:54 -07:00
test_router fix(logging): stop pinning large request payloads past request end (#33455) 2026-07-15 15:28:37 -07:00
types perf(core): fast-path SafeAttributeModel.__delattr__ for declared fields (#33993) 2026-07-21 00:06:56 +00:00
vector_stores style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
videos test(videos): add 1:1 test file scaffold for videos component paths (#30631) 2026-06-29 09:12:51 +05:30
__init__.py
conftest.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
log.txt
readme.md
test__types.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_a2a_registry_lookup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_acompletion_session_reuse_e2e.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_add_deployment_no_master_key.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_aembedding_session_reuse_e2e.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_anthropic_beta_headers_filtering.py chore(oss): litellm oss staging 120626 (#30292) 2026-06-12 09:49:25 -07:00
test_anthropic_skills_transformation.py fix: encode upstream URL path identifiers 2026-04-29 22:02:39 -07:00
test_anthropic_sonnet_1hr_cache_pricing.py chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_audio_transcription_rust_bridge.py perf(bedrock): audio transcription via rust core (py->rust bridge) (#33990) 2026-07-20 14:09:41 -07:00
test_azure_ad_token_credential_resolution.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_azure_video_router.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_batch_completion_models_all_responses.py test(batches): move orphan tests into tests/test_litellm for CI coverage (#30510) 2026-06-16 10:20:59 -07:00
test_bedrock_anthropic_1hr_cache_pricing.py Litellm oss staging 030626 (#29578) 2026-06-03 11:01:51 -07:00
test_bedrock_usgov_haiku_1hr_cache.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_bedrock_usgov_pricing.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_budget_ratchet_check.py refactor(lint): collapse type/lint budgets to a single per-rule limit (#31883) 2026-07-01 18:12:35 +03:00
test_chat_ui_responses_session.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_check_licenses.py fix(check_licenses): read PEP 639 license-expression metadata (#28529) 2026-05-22 11:22:38 -07:00
test_check_type_discipline.py chore(lint): zero out crash-class pyright rules and ban new type: ignore comments (#32152) 2026-07-04 16:56:12 -07:00
test_circleci_path_filter.py ci: gate CircleCI jobs on changed paths (#32080) 2026-07-04 19:15:08 -07:00
test_claude_fable_5_config.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_claude_haiku_4_5_config.py feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238) 2026-05-28 18:50:33 -07:00
test_claude_opus_4_6_config.py fix(azure_ai): advertise 1M context window for Claude Opus 4.6+ on Foundry 2026-07-24 21:53:12 +00:00
test_claude_opus_4_8_config.py fix(azure_ai): advertise 1M context window for Claude Opus 4.6+ on Foundry 2026-07-24 21:53:12 +00:00
test_claude_opus_5_config.py feat(anthropic): add Claude Opus 5 2026-07-24 10:43:49 -07:00
test_claude_sonnet_4_6_config.py feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238) 2026-05-28 18:50:33 -07:00
test_claude_sonnet_5_config.py fix(model_prices): apply claude-sonnet-5 introductory pricing through 2026-08-31 (#31917) 2026-07-01 17:45:57 -07:00
test_cloudflare_workers_ai_model_metadata.py feat(cloudflare): add current Workers AI text-generation models to the cost map (#31051) 2026-06-23 10:44:37 -07:00
test_command_r7b_pricing.py chore: litellm oss 170626 (#30637) 2026-06-17 21:11:12 -07:00
test_completion_timeout_resolution.py fix(router): honor litellm_settings.request_timeout as an independent per-attempt timeout (#31119) 2026-06-23 14:22:54 -07:00
test_compression.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
test_constants.py
test_container_router.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_cost_calculation_log_level.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_cost_calculator.py feat(dashscope): add qwen3.7-plus and qwen3.7-max to the model cost map (#35123) 2026-07-29 14:22:22 -07:00
test_count_tokens_public_api.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_dashscope_image_generation.py fix(dashscope): fail fast on image generation API errors 2026-04-23 18:41:01 +05:30
test_deepseek_model_metadata.py Revert "Merge pull request #16590 from Chesars/refactor/remove-backup-file-dry-principle" 2026-04-25 17:10:41 -03:00
test_dockerfile_non_root.py fix(docker.non_root): use numeric UID 65534 for K8s runAsNonRoot (#26268) 2026-04-22 18:00:04 -07:00
test_eager_tiktoken_load.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_exports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_header_preservation.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_mapping_request_attribute.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_filter_out_litellm_params.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_get_blog_posts.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_git_hooks.py chore(hooks): enforce Conventional Commits and Conventional Branches (#30174) 2026-06-11 10:00:23 -07:00
test_github_close_low_quality_prs.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_github_review_gate.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_github_triage_with_llm.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_github_triage_workflows.py fix(test): drop references to removed Agent Shin workflows (#30791) 2026-06-18 15:47:51 -07:00
test_gpt_5_5_model_metadata.py chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_gpt_5_6_model_metadata.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_gpt_image_cost_calculator.py chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
test_gpt_realtime_mode.py test(pricing): pin the realtime mode assertion to the bundled cost map (#33806) 2026-07-18 02:52:55 +00:00
test_groq_streaming_encoding.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_guardrail_exception_status_codes.py Litellm oss staging (#28161) 2026-05-18 16:27:44 -07:00
test_lazy_imports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_litellm_params_reserved_keys.py
test_logging.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_lowest_latency_zero_tokens.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_main.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_main_module_header.py Add main module header comment 2026-05-06 00:26:17 +00:00
test_mistral_medium_3_5_model_metadata.py fix(cost-map): retarget mistral-medium-latest to Medium 3.5 and add date-pinned aliases (#31373) 2026-06-25 18:27:18 -07:00
test_model_block_unblock.py fix(proxy): report when a model write does not survive the post-write reload 2026-07-28 18:52:06 -07:00
test_model_cost_aliases.py
test_model_param_helper.py style: black format test_model_param_helper.py 2026-04-15 18:18:52 -07:00
test_model_prices_schema.py ci: enforce format assertions so calendar-impossible deprecation dates fail validation 2026-07-28 16:11:22 -07:00
test_model_response_normalization.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_muse_spark_1_1_model_metadata.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_nested_drop_params.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_openai_embedding_encoding_format_default.py fix(embeddings): allow omitting encoding_format via env sentinel none 2026-05-01 23:03:13 +05:30
test_prisma_generate_if_needed.py perf(lint): skip and cache base gate passes, parallelize make lint, skip redundant prisma generate (#32000) 2026-07-02 19:24:00 -07:00
test_project_alias_tracking.py
test_project_tags_pydantic.py
test_rag_openai_ingestion.py chore: litellm oss staging (#30745) 2026-06-18 13:55:35 -07:00
test_rate_limit_error_unification.py feat: standardize rate limit errors with category, rate_limit_type, model, and llm_provider fields (#27687) 2026-06-06 17:50:29 -07:00
test_redact_string_in_error_paths.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_redis.py fix(redis): honor ssl value instead of key presence when building async connection pool (#32590) 2026-07-16 13:38:01 -07:00
test_register_model_custom_pricing.py fix(utils): resolve bedrock regional inference profiles to regional pricing in get_model_info (LIT-4056) (#32389) 2026-07-07 20:49:03 -07:00
test_register_model_zero_cost_persistence.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_responses_api_bridge_non_stream.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_responses_id_security.py fix(responses_id_security): decrypt response ids for input_items follow-ups (#32269) 2026-07-06 14:03:15 -07:00
test_responses_streaming_container_ownership.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_retrieve_batch_bedrock_dispatch.py feat(batch-job): bedrock batch model invocation job retrieval (#26834) 2026-05-11 13:22:26 -07:00
test_router.py Merge pull request #35028 from BerriAI/litellm_batch_provider_credentials 2026-07-29 11:59:09 -07:00
test_router_block_helpers.py feat: litellm oss 110626 (#30202) 2026-06-11 22:30:26 -07:00
test_router_exception_redaction.py fix(router): mask provider credentials embedded in fallback error messages (#32083) 2026-07-03 18:48:06 -07:00
test_router_google_genai.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_model_cost_isolation.py fix(router): propagate capability flags to shared backend cost map key 2026-07-20 16:46:30 -07:00
test_router_order_fallback.py chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_router_per_deployment_num_retries.py fix(router): honor request-level num_retries over global litellm_settings.num_retries (#34124) 2026-07-21 21:32:34 +00:00
test_router_redis_init.py
test_router_retry_non_retryable_errors.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_retry_policy_update.py fix(proxy): tighten role gating on /get/config/callbacks response (#31745) 2026-07-01 17:58:31 -07:00
test_router_silent_experiment.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_streaming_fallback_metadata.py chore: litellm oss staging (#30968) 2026-06-23 07:31:44 -07:00
test_router_weighted_failover.py feat: add weighted-routing failover (#27980) 2026-05-15 17:28:54 +00:00
test_ruff_strict_gate.py perf(lint): skip and cache base gate passes, parallelize make lint, skip redundant prisma generate (#32000) 2026-07-02 19:24:00 -07:00
test_secret_redaction.py fix(key management): enforce minimum custom key length and mask short keys in key_name (#33462) 2026-07-15 18:27:38 -07:00
test_service_logger.py fix: missing span for guardrail passthrough (#29552) 2026-06-03 01:25:15 +00:00
test_setup_wizard.py
test_shared_session_integration.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_ssl_verify_unit.py refactor(bedrock): remove the dead BedrockLLM invoke code path 2026-07-29 20:25:36 -07:00
test_stream_chunk_builder_annotations.py
test_streaming_connection_cleanup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_system_message_format_bug.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_thinking_enabled.py Litellm oss staging 250526 (#28770) 2026-05-26 11:57:39 -07:00
test_triage_rollout_heads_up.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_type_check_gate.py perf(lint): skip and cache base gate passes, parallelize make lint, skip redundant prisma generate (#32000) 2026-07-02 19:24:00 -07:00
test_type_discipline_gate.py refactor(lint): collapse type/lint budgets to a single per-rule limit (#31883) 2026-07-01 18:12:35 +03:00
test_utils.py fix: match exact class in callback dedup so a custom subclass does not block a built-in logger (#34804) 2026-07-27 12:18:42 -07:00
test_utils_module_docstring.py Add utils module docstring 2026-05-06 00:42:49 +00:00
test_uuid_helper.py
test_vcr_safe_body_matcher.py test: stabilize batch VCR coverage and stop live upload/network leaks (#29477) 2026-06-02 16:11:52 -07:00
test_video_generation.py fix(vertex-ai): use DB credentials in video handlers + implement Veo video edit (#29098) 2026-05-28 11:45:41 -07:00
test_xai_grok_4_3_model_metadata.py feat(xai): add grok-4.3 and grok-4.3-latest to model_prices_and_conte… (#27154) 2026-05-07 09:06:56 -07:00
test_xai_responses_auto_routing.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py