scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-13 23:11:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 101
litellm/tests/test_litellm
History
Exact
Tin Chi Lo 6517b1a764 fix(mcp): carry preview length warnings for OpenAPI specs and the form alias 
Per review: the create-server preview request now includes the alias the
user typed so warnings measure the prefix the runtime will actually apply,
and the OpenAPI spec preview branch returns the same warnings array as the
MCP branch since registered OpenAPI tools get the server prefix too
2026-07-07 14:02:59 -07:00
..
a2a_protocol fix(responses): stop scheduling sync success_handler concurrently with async_success_handler (#32239) 2026-07-07 09:13:50 -07:00
anthropic_interface/exceptions style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
batches fix(batches): price anthropic passthrough message batches correctly in batch cost job (#32307) 2026-07-06 20:33:57 -07:00
caching fix(caching): pass only metadata to valkey semantic async embedding (#32295) 2026-07-07 08:52:03 +03:00
completion_extras chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
containers fix(containers): record ownership for service-account keys + fix Prisma Json serialization (#28990) 2026-05-27 21:00:07 -07:00
enterprise fix: prevent duplicate budget alert emails on concurrent threshold crossings (#32011) 2026-07-03 14:59:34 -07:00
expected_fine_tuning_api
expected_responses_api_request
experimental_mcp_client fix(mcp): stop logging tool-call input in MCP client (#31393) 2026-06-26 17:42:05 -07:00
google_genai chore: litellm oss staging 250626 (#31305) 2026-06-25 21:00:28 -07:00
images fix(image_edit): read vertex_project/location from litellm_params in Imagen get_complete_url 2026-04-21 15:03:40 +05:30
integrations feat(otel): stamp gen_ai.response.time_to_first_chunk on streaming LLM spans (#32236) 2026-07-07 09:15:49 -07:00
interactions fix(responses): stop scheduling sync success_handler concurrently with async_success_handler (#32239) 2026-07-07 09:13:50 -07:00
litellm_core_utils fix(responses): stop scheduling sync success_handler concurrently with async_success_handler (#32239) 2026-07-07 09:13:50 -07:00
llms fix(vertex_ai): build full request path when custom api_base has no path (#32367) 2026-07-07 13:17:07 -07:00
models feat(litellm): add models and repository layers (#29686) 2026-06-06 20:59:33 -07:00
ocr feat(ocr): thin Rust OCR Python bridge (#31368) 2026-06-25 18:42:59 -07:00
passthrough fix(logging): classify allm_passthrough_route as async to prevent duplicate success callbacks (#32265) 2026-07-06 15:04:05 -07:00
proxy fix(mcp): carry preview length warnings for OpenAPI specs and the form alias 2026-07-07 14:02:59 -07:00
realtime_api fix(realtime): preserve nested transcription model and session-first model priority 2026-07-03 14:21:22 -07:00
repositories fix(proxy): count only active users toward license seat limit (#31227) 2026-06-29 18:01:02 -07:00
responses fix(responses): stop scheduling sync success_handler concurrently with async_success_handler (#32239) 2026-07-07 09:13:50 -07:00
router_strategy feat(complexity_router): add custom_technical_keywords config (#32262) 2026-07-06 13:00:30 -07:00
router_utils feat(router): add separate ITPM/OTPM deployment rate limits (#31952) 2026-07-05 21:58:35 +05:30
sandbox feat: add opensandbox sandbox provider (#31024) 2026-06-23 09:05:13 -07:00
secret_managers chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_router feat(router): add separate ITPM/OTPM deployment rate limits (#31952) 2026-07-05 21:58:35 +05:30
types fix(anthropic): drop unsignable thinking blocks and allow null signature in logging (LIT-4007) (#31654) 2026-06-29 21:01:03 -07:00
vector_stores style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
videos test(videos): add 1:1 test file scaffold for videos component paths (#30631) 2026-06-29 09:12:51 +05:30
__init__.py
conftest.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
log.txt
readme.md
test__types.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_a2a_registry_lookup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_acompletion_session_reuse_e2e.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_add_deployment_no_master_key.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_aembedding_session_reuse_e2e.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_anthropic_beta_headers_filtering.py chore(oss): litellm oss staging 120626 (#30292) 2026-06-12 09:49:25 -07:00
test_anthropic_skills_transformation.py fix: encode upstream URL path identifiers 2026-04-29 22:02:39 -07:00
test_anthropic_sonnet_1hr_cache_pricing.py chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_azure_ad_token_credential_resolution.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_azure_video_router.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_batch_completion_models_all_responses.py test(batches): move orphan tests into tests/test_litellm for CI coverage (#30510) 2026-06-16 10:20:59 -07:00
test_bedrock_anthropic_1hr_cache_pricing.py Litellm oss staging 030626 (#29578) 2026-06-03 11:01:51 -07:00
test_bedrock_usgov_haiku_1hr_cache.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_bedrock_usgov_pricing.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_budget_ratchet_check.py refactor(lint): collapse type/lint budgets to a single per-rule limit (#31883) 2026-07-01 18:12:35 +03:00
test_chat_ui_responses_session.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_check_licenses.py fix(check_licenses): read PEP 639 license-expression metadata (#28529) 2026-05-22 11:22:38 -07:00
test_check_type_discipline.py chore(lint): zero out crash-class pyright rules and ban new type: ignore comments (#32152) 2026-07-04 16:56:12 -07:00
test_circleci_path_filter.py ci: gate CircleCI jobs on changed paths (#32080) 2026-07-04 19:15:08 -07:00
test_claude_fable_5_config.py Add Claude Fable 5 across Anthropic, Bedrock, Vertex AI, and Azure AI (#30064) 2026-06-10 08:50:15 +05:30
test_claude_haiku_4_5_config.py feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238) 2026-05-28 18:50:33 -07:00
test_claude_opus_4_6_config.py feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238) 2026-05-28 18:50:33 -07:00
test_claude_opus_4_8_config.py fix(anthropic): route Claude Opus 4.8 through adaptive thinking (#29702) 2026-06-05 16:19:01 +05:30
test_claude_sonnet_4_6_config.py feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238) 2026-05-28 18:50:33 -07:00
test_claude_sonnet_5_config.py fix(model_prices): apply claude-sonnet-5 introductory pricing through 2026-08-31 (#31917) 2026-07-01 17:45:57 -07:00
test_cloudflare_workers_ai_model_metadata.py feat(cloudflare): add current Workers AI text-generation models to the cost map (#31051) 2026-06-23 10:44:37 -07:00
test_command_r7b_pricing.py chore: litellm oss 170626 (#30637) 2026-06-17 21:11:12 -07:00
test_completion_timeout_resolution.py fix(router): honor litellm_settings.request_timeout as an independent per-attempt timeout (#31119) 2026-06-23 14:22:54 -07:00
test_compression.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
test_constants.py
test_container_router.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_cost_calculation_log_level.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_cost_calculator.py fix(batches): price anthropic passthrough message batches correctly in batch cost job (#32307) 2026-07-06 20:33:57 -07:00
test_count_tokens_public_api.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_dashscope_image_generation.py fix(dashscope): fail fast on image generation API errors 2026-04-23 18:41:01 +05:30
test_deepseek_model_metadata.py Revert "Merge pull request #16590 from Chesars/refactor/remove-backup-file-dry-principle" 2026-04-25 17:10:41 -03:00
test_dockerfile_non_root.py fix(docker.non_root): use numeric UID 65534 for K8s runAsNonRoot (#26268) 2026-04-22 18:00:04 -07:00
test_eager_tiktoken_load.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_exports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_header_preservation.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_mapping_request_attribute.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_filter_out_litellm_params.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_get_blog_posts.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_git_hooks.py chore(hooks): enforce Conventional Commits and Conventional Branches (#30174) 2026-06-11 10:00:23 -07:00
test_github_close_low_quality_prs.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_github_review_gate.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_github_triage_with_llm.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_github_triage_workflows.py fix(test): drop references to removed Agent Shin workflows (#30791) 2026-06-18 15:47:51 -07:00
test_gpt_5_5_model_metadata.py chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_gpt_image_cost_calculator.py chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
test_groq_streaming_encoding.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_guardrail_exception_status_codes.py Litellm oss staging (#28161) 2026-05-18 16:27:44 -07:00
test_lazy_imports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_litellm_params_reserved_keys.py
test_logging.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_lowest_latency_zero_tokens.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_main.py fix(main): forward verbosity param to chat completion providers (#32254) 2026-07-06 10:41:49 -07:00
test_main_module_header.py Add main module header comment 2026-05-06 00:26:17 +00:00
test_mistral_medium_3_5_model_metadata.py fix(cost-map): retarget mistral-medium-latest to Medium 3.5 and add date-pinned aliases (#31373) 2026-06-25 18:27:18 -07:00
test_model_block_unblock.py feat: litellm oss 110626 (#30202) 2026-06-11 22:30:26 -07:00
test_model_cost_aliases.py
test_model_param_helper.py style: black format test_model_param_helper.py 2026-04-15 18:18:52 -07:00
test_model_response_normalization.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_nested_drop_params.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_openai_embedding_encoding_format_default.py fix(embeddings): allow omitting encoding_format via env sentinel none 2026-05-01 23:03:13 +05:30
test_prisma_generate_if_needed.py perf(lint): skip and cache base gate passes, parallelize make lint, skip redundant prisma generate (#32000) 2026-07-02 19:24:00 -07:00
test_project_alias_tracking.py
test_project_tags_pydantic.py
test_rag_openai_ingestion.py chore: litellm oss staging (#30745) 2026-06-18 13:55:35 -07:00
test_rate_limit_error_unification.py feat: standardize rate limit errors with category, rate_limit_type, model, and llm_provider fields (#27687) 2026-06-06 17:50:29 -07:00
test_redact_string_in_error_paths.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_redis.py fix(redis): re-establish async cluster connections after a node restart (#31577) 2026-06-30 12:25:15 -07:00
test_register_model_custom_pricing.py fix(main): stop per-request custom pricing from clobbering shared model_cost pricing (#32163) 2026-07-07 10:25:31 -07:00
test_register_model_zero_cost_persistence.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_responses_api_bridge_non_stream.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_responses_id_security.py fix(responses_id_security): decrypt response ids for input_items follow-ups (#32269) 2026-07-06 14:03:15 -07:00
test_responses_streaming_container_ownership.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_retrieve_batch_bedrock_dispatch.py feat(batch-job): bedrock batch model invocation job retrieval (#26834) 2026-05-11 13:22:26 -07:00
test_router.py chore(router): simplify unknown-model error message construction 2026-06-28 21:13:19 +00:00
test_router_block_helpers.py feat: litellm oss 110626 (#30202) 2026-06-11 22:30:26 -07:00
test_router_exception_redaction.py fix(router): mask provider credentials embedded in fallback error messages (#32083) 2026-07-03 18:48:06 -07:00
test_router_google_genai.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_model_cost_isolation.py fix(main): stop per-request custom pricing from clobbering shared model_cost pricing (#32163) 2026-07-07 10:25:31 -07:00
test_router_order_fallback.py chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_router_per_deployment_num_retries.py fix(router): guard num_retries=None in async_function_with_retries (#30036) 2026-06-23 10:16:15 -07:00
test_router_redis_init.py
test_router_retry_non_retryable_errors.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_retry_policy_update.py fix(proxy): tighten role gating on /get/config/callbacks response (#31745) 2026-07-01 17:58:31 -07:00
test_router_silent_experiment.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_streaming_fallback_metadata.py chore: litellm oss staging (#30968) 2026-06-23 07:31:44 -07:00
test_router_weighted_failover.py feat: add weighted-routing failover (#27980) 2026-05-15 17:28:54 +00:00
test_ruff_strict_gate.py perf(lint): skip and cache base gate passes, parallelize make lint, skip redundant prisma generate (#32000) 2026-07-02 19:24:00 -07:00
test_secret_redaction.py Litellm oss staging (#29492) 2026-06-02 08:48:10 -07:00
test_service_logger.py fix: missing span for guardrail passthrough (#29552) 2026-06-03 01:25:15 +00:00
test_setup_wizard.py
test_shared_session_integration.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_ssl_verify_unit.py Litellm OSS Staging (#29161) 2026-06-01 21:22:35 -07:00
test_stream_chunk_builder_annotations.py
test_streaming_connection_cleanup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_system_message_format_bug.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_thinking_enabled.py Litellm oss staging 250526 (#28770) 2026-05-26 11:57:39 -07:00
test_triage_rollout_heads_up.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_type_check_gate.py perf(lint): skip and cache base gate passes, parallelize make lint, skip redundant prisma generate (#32000) 2026-07-02 19:24:00 -07:00
test_type_discipline_gate.py refactor(lint): collapse type/lint budgets to a single per-rule limit (#31883) 2026-07-01 18:12:35 +03:00
test_utils.py feat(tencent): add Tencent TokenHub as a provider (#31903) 2026-07-02 18:31:59 -07:00
test_utils_module_docstring.py Add utils module docstring 2026-05-06 00:42:49 +00:00
test_uuid_helper.py
test_vcr_safe_body_matcher.py test: stabilize batch VCR coverage and stop live upload/network leaks (#29477) 2026-06-02 16:11:52 -07:00
test_video_generation.py fix(vertex-ai): use DB credentials in video handlers + implement Veo video edit (#29098) 2026-05-28 11:45:41 -07:00
test_xai_grok_4_3_model_metadata.py feat(xai): add grok-4.3 and grok-4.3-latest to model_prices_and_conte… (#27154) 2026-05-07 09:06:56 -07:00
test_xai_responses_auto_routing.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py