scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-09 03:18:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 109
litellm/tests/test_litellm
History
Yassin Kortam fc644cff3d
perf(spend-logs): only strip NUL bytes in safe_dumps when present (#31424) 
safe_dumps ran strip_null_bytes (a str.replace) on every string value and
every dict key during recursive serialization. NUL bytes are vanishingly
rare, so for the common case this was pure overhead that scaled with payload
size; with store_prompts_in_spend_logs the full prompt and response are
serialized on every request, so it landed directly in the per-request hot
path. Guard the strip behind a cheap "\x00" in obj membership check so
NUL-free strings are returned untouched. Behavior is unchanged: NUL bytes are
still stripped from values, keys, nested structures, and the str() fallback.
2026-06-26 09:37:19 -07:00
..
a2a_protocol fix(a2a): forward agent_extra_headers through completion bridge (#28277) 2026-06-11 21:56:18 -07:00
anthropic_interface/exceptions
caching chore: litellm oss staging 250626 (#31305) 2026-06-25 21:00:28 -07:00
completion_extras chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
containers fix(containers): record ownership for service-account keys + fix Prisma Json serialization (#28990) 2026-05-27 21:00:07 -07:00
enterprise chore: litellm oss staging (#30745) 2026-06-18 13:55:35 -07:00
expected_fine_tuning_api
expected_responses_api_request
experimental_mcp_client feat(mcp): graft v2 resolver onto _create_mcp_client (none + api_key static family) (#31058) 2026-06-24 14:53:33 -07:00
google_genai chore: litellm oss staging 250626 (#31305) 2026-06-25 21:00:28 -07:00
images
integrations chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
interactions chore: litellm oss staging (#30968) 2026-06-23 07:31:44 -07:00
litellm_core_utils perf(spend-logs): only strip NUL bytes in safe_dumps when present (#31424) 2026-06-26 09:37:19 -07:00
llms test: add realtime proxy e2e suite across providers (#30960) 2026-06-26 09:36:49 -07:00
models feat(litellm): add models and repository layers (#29686) 2026-06-06 20:59:33 -07:00
ocr feat(ocr): thin Rust OCR Python bridge (#31368) 2026-06-25 18:42:59 -07:00
passthrough refactor: trim explanatory comments from streaming-flush fix 2026-04-30 02:39:28 +00:00
proxy chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
repositories feat(litellm): add models and repository layers (#29686) 2026-06-06 20:59:33 -07:00
responses chore: litellm oss staging 250626 (#31305) 2026-06-25 21:00:28 -07:00
router_strategy chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
router_utils chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
sandbox feat: add opensandbox sandbox provider (#31024) 2026-06-23 09:05:13 -07:00
secret_managers chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_router
types refactor(completion): extract provider dispatch into typed helpers so basedpyright can analyze it (#30813) 2026-06-23 07:29:31 -07:00
vector_stores
__init__.py
conftest.py
log.txt
readme.md
test__types.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_a2a_registry_lookup.py
test_acompletion_session_reuse_e2e.py
test_add_deployment_no_master_key.py
test_aembedding_session_reuse_e2e.py
test_anthropic_beta_headers_filtering.py chore(oss): litellm oss staging 120626 (#30292) 2026-06-12 09:49:25 -07:00
test_anthropic_skills_transformation.py fix: encode upstream URL path identifiers 2026-04-29 22:02:39 -07:00
test_anthropic_sonnet_1hr_cache_pricing.py chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_azure_ad_token_credential_resolution.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_azure_video_router.py
test_batch_completion_models_all_responses.py test(batches): move orphan tests into tests/test_litellm for CI coverage (#30510) 2026-06-16 10:20:59 -07:00
test_bedrock_anthropic_1hr_cache_pricing.py Litellm oss staging 030626 (#29578) 2026-06-03 11:01:51 -07:00
test_bedrock_usgov_haiku_1hr_cache.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_bedrock_usgov_pricing.py Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
test_budget_ratchet_check.py chore(lint): widen ANN slack to 10% of baseline and drop PLR0913 from the strict gate (#31335) 2026-06-25 14:43:45 -07:00
test_chat_ui_responses_session.py
test_check_licenses.py fix(check_licenses): read PEP 639 license-expression metadata (#28529) 2026-05-22 11:22:38 -07:00
test_check_type_discipline.py ci(lint): ratcheted type-discipline gate (mutable collections, casts, guards, kwargs, suppressions) (#30500) 2026-06-16 16:59:21 -07:00
test_claude_fable_5_config.py Add Claude Fable 5 across Anthropic, Bedrock, Vertex AI, and Azure AI (#30064) 2026-06-10 08:50:15 +05:30
test_claude_haiku_4_5_config.py feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238) 2026-05-28 18:50:33 -07:00
test_claude_opus_4_6_config.py feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238) 2026-05-28 18:50:33 -07:00
test_claude_opus_4_8_config.py fix(anthropic): route Claude Opus 4.8 through adaptive thinking (#29702) 2026-06-05 16:19:01 +05:30
test_claude_sonnet_4_6_config.py feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (#29238) 2026-05-28 18:50:33 -07:00
test_cloudflare_workers_ai_model_metadata.py feat(cloudflare): add current Workers AI text-generation models to the cost map (#31051) 2026-06-23 10:44:37 -07:00
test_command_r7b_pricing.py chore: litellm oss 170626 (#30637) 2026-06-17 21:11:12 -07:00
test_completion_timeout_resolution.py fix(router): honor litellm_settings.request_timeout as an independent per-attempt timeout (#31119) 2026-06-23 14:22:54 -07:00
test_compression.py
test_constants.py
test_container_router.py
test_cost_calculation_log_level.py
test_cost_calculator.py fix(realtime): stop revalidating realtime events at the logging boundary (#31054) 2026-06-22 20:43:03 -07:00
test_count_tokens_public_api.py
test_dashscope_image_generation.py
test_deepseek_model_metadata.py Revert "Merge pull request #16590 from Chesars/refactor/remove-backup-file-dry-principle" 2026-04-25 17:10:41 -03:00
test_dockerfile_non_root.py
test_eager_tiktoken_load.py
test_exception_exports.py
test_exception_header_preservation.py
test_exception_mapping_request_attribute.py
test_filter_out_litellm_params.py
test_get_blog_posts.py
test_git_hooks.py chore(hooks): enforce Conventional Commits and Conventional Branches (#30174) 2026-06-11 10:00:23 -07:00
test_github_close_low_quality_prs.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_github_review_gate.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_github_triage_with_llm.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_github_triage_workflows.py fix(test): drop references to removed Agent Shin workflows (#30791) 2026-06-18 15:47:51 -07:00
test_gpt_5_5_model_metadata.py chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_gpt_image_cost_calculator.py chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
test_groq_streaming_encoding.py
test_guardrail_exception_status_codes.py Litellm oss staging (#28161) 2026-05-18 16:27:44 -07:00
test_lazy_imports.py
test_litellm_params_reserved_keys.py
test_logging.py
test_lowest_latency_zero_tokens.py
test_main.py Litellm oss staging 040626 (#29671) 2026-06-04 11:07:20 -07:00
test_main_module_header.py Add main module header comment 2026-05-06 00:26:17 +00:00
test_mistral_medium_3_5_model_metadata.py fix(cost-map): retarget mistral-medium-latest to Medium 3.5 and add date-pinned aliases (#31373) 2026-06-25 18:27:18 -07:00
test_model_block_unblock.py feat: litellm oss 110626 (#30202) 2026-06-11 22:30:26 -07:00
test_model_cost_aliases.py
test_model_param_helper.py
test_model_response_normalization.py
test_nested_drop_params.py
test_openai_embedding_encoding_format_default.py fix(embeddings): allow omitting encoding_format via env sentinel none 2026-05-01 23:03:13 +05:30
test_project_alias_tracking.py
test_project_tags_pydantic.py
test_rag_openai_ingestion.py chore: litellm oss staging (#30745) 2026-06-18 13:55:35 -07:00
test_rate_limit_error_unification.py feat: standardize rate limit errors with category, rate_limit_type, model, and llm_provider fields (#27687) 2026-06-06 17:50:29 -07:00
test_redact_string_in_error_paths.py
test_redis.py Litellm oss staging (#28161) 2026-05-18 16:27:44 -07:00
test_register_model_custom_pricing.py fix(register_model): preserve built-in cache pricing when registering custom overrides under unmapped keys (#30044) 2026-06-10 12:11:03 -07:00
test_register_model_zero_cost_persistence.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_responses_api_bridge_non_stream.py
test_responses_id_security.py
test_responses_streaming_container_ownership.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_retrieve_batch_bedrock_dispatch.py feat(batch-job): bedrock batch model invocation job retrieval (#26834) 2026-05-11 13:22:26 -07:00
test_router.py fix(router): surface clean RateLimitError on mid-stream 429 with no fallbacks (#31298) 2026-06-25 23:37:29 -07:00
test_router_block_helpers.py feat: litellm oss 110626 (#30202) 2026-06-11 22:30:26 -07:00
test_router_exception_redaction.py chore: litellm oss 170626 (#30637) 2026-06-17 21:11:12 -07:00
test_router_google_genai.py
test_router_model_cost_isolation.py fix(router): isolate all per-deployment pricing overrides from sibling deployments (#31021) 2026-06-23 00:11:22 -07:00
test_router_order_fallback.py chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_router_per_deployment_num_retries.py fix(router): guard num_retries=None in async_function_with_retries (#30036) 2026-06-23 10:16:15 -07:00
test_router_redis_init.py
test_router_retry_non_retryable_errors.py
test_router_silent_experiment.py
test_router_streaming_fallback_metadata.py chore: litellm oss staging (#30968) 2026-06-23 07:31:44 -07:00
test_router_weighted_failover.py feat: add weighted-routing failover (#27980) 2026-05-15 17:28:54 +00:00
test_ruff_strict_gate.py feat(auth): resolve caller identity once into a Principal at the auth seam (#30887) 2026-06-20 18:49:41 -07:00
test_secret_redaction.py Litellm oss staging (#29492) 2026-06-02 08:48:10 -07:00
test_service_logger.py fix: missing span for guardrail passthrough (#29552) 2026-06-03 01:25:15 +00:00
test_setup_wizard.py
test_shared_session_integration.py
test_ssl_verify_unit.py Litellm OSS Staging (#29161) 2026-06-01 21:22:35 -07:00
test_stream_chunk_builder_annotations.py
test_streaming_connection_cleanup.py
test_system_message_format_bug.py
test_thinking_enabled.py Litellm oss staging 250526 (#28770) 2026-05-26 11:57:39 -07:00
test_triage_rollout_heads_up.py feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle (#30433) 2026-06-17 20:42:27 -07:00
test_type_check_gate.py ci: make the basedpyright budget gate delta-vs-base (#31106) 2026-06-23 11:10:05 -07:00
test_type_discipline_gate.py ci(lint): ratcheted type-discipline gate (mutable collections, casts, guards, kwargs, suppressions) (#30500) 2026-06-16 16:59:21 -07:00
test_utils.py test: add realtime proxy e2e suite across providers (#30960) 2026-06-26 09:36:49 -07:00
test_utils_module_docstring.py Add utils module docstring 2026-05-06 00:42:49 +00:00
test_uuid_helper.py
test_vcr_safe_body_matcher.py test: stabilize batch VCR coverage and stop live upload/network leaks (#29477) 2026-06-02 16:11:52 -07:00
test_video_generation.py fix(vertex-ai): use DB credentials in video handlers + implement Veo video edit (#29098) 2026-05-28 11:45:41 -07:00
test_xai_grok_4_3_model_metadata.py feat(xai): add grok-4.3 and grok-4.3-latest to model_prices_and_conte… (#27154) 2026-05-07 09:06:56 -07:00
test_xai_responses_auto_routing.py

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py