scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-09 03:18:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 108
litellm/tests/test_litellm
History
mateo-berri b604e2b20c refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 
About 35,000 fixes ruff marks safe across 32 rules (UP006/UP045/UP007
modern annotations, UP032 f-strings, SIM114/SIM118, RET501, and
friends), removal of the 1,296 typing imports the rewrite orphaned, and
hand fixes for what the fixers could not see: five star-import
freeloaders of typing names, two F823 late-import annotations, the
/get/config/list introspection crash on types.UnionType, redundant
function-local RoleMappings imports in ui_sso.py that shadowed the
module-level name once the annotation lost its quotes, and one FURB168
tautology.

B009/B010/PIE804/RUF019 are excluded on purpose: their safe fixes
rewrite getattr/setattr/**-splat/key-in-dict escape hatches into forms
basedpyright then rejects (283 new errors measured), so their budgets
stay at base values.

ruff-strict-budget.json drops by 39,579 this commit (39,968 across the
branch) with 28 rules at an actual 0 and 9 more sharply down.
type-discipline-budget.json ratchets LIT002/LIT006/LIT009 down; LIT001
moves to the now-honest total: the checker matches the spelling `set`
but not the alias `Set`, so the 160 typing.Set annotations rewritten to
set[...] were always mutable-set annotations and only now count.
2026-08-01 15:43:29 -07:00
..
a2a_protocol fix(otel): label retrieval and agent metrics correctly and emit gen_ai.provider.name (#35151) 2026-07-30 13:48:59 -07:00
anthropic_interface fix(rust): route agentic-completion-hook /messages requests to Python for all stream modes (#34126) 2026-07-22 00:36:47 +00:00
batches Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_batch_output_single_pass 2026-07-30 10:55:28 -07:00
caching fix(caching): stamp provider on embedding cache-hit logs so spend logs record provider 2026-07-30 22:11:01 +00:00
completion_extras chore: merge litellm_internal_staging into litellm_fix_responses_bridge_streaming_contract 2026-07-25 00:20:31 +00:00
compression fix(guardrails/headroom): stop compressing the turn the model must act on (#35294) 2026-07-30 18:53:31 -07:00
containers
enterprise test(managed-files): call store_unified_file_id twice and assert upsert payloads 2026-07-29 18:51:25 -07:00
expected_fine_tuning_api
expected_responses_api_request test(responses): replace perma-skip azure shell e2e with offline coverage (#32444) 2026-07-08 10:01:41 -07:00
experimental_mcp_client fix(mcp): sanitize Anthropic tool schemas and stop encoding gateway names 2026-07-17 10:33:28 -07:00
google_genai
images
integrations feat(s3): support SSE-KMS encryption params on both S3 logging paths (#35291) 2026-07-30 21:18:33 -07:00
interactions fix(tests): assert Content variants are identified by type, not by the discriminator keyword (#35161) 2026-07-29 17:49:30 -07:00
litellm_core_utils Merge pull request #35324 from BerriAI/litellm_gpt56_pricing_test_gaps 2026-07-30 22:23:27 -07:00
llms Merge pull request #35004 from mgeorgaklis/fix/gemini-thought-signature-duplication 2026-07-31 11:51:18 -07:00
models
ocr test: litellm fix failing tests (#32577) 2026-07-09 13:54:45 -07:00
passthrough fix(passthrough): stream non-sse passthrough responses instead of buffering in memory (#32386) 2026-07-07 20:51:15 -07:00
proxy refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
rag fix(rag): track LLM completion usage and spend for /v1/rag/query (#32438) 2026-07-17 17:45:27 +00:00
realtime_api
repositories test(repositories): lock in UserRepository JSON column decoding 2026-07-31 19:26:04 +00:00
rerank_api fix(rerank): log optional_rerank_params at debug to stop leaking request content (#32533) 2026-07-08 18:48:03 -07:00
responses fix(responses): map all documented in-stream error codes to real HTTP statuses 2026-07-30 19:17:32 -07:00
router_strategy feat(spend-logs): record when a spend log row is the auto-router's own classifier call (#35300) 2026-07-30 19:26:42 -07:00
router_utils fix(proxy): reject model writes that corrupt an auto-router pseudo-model 2026-07-28 20:25:07 -07:00
sandbox
secret_managers fix(bedrock): stop replaying expired Google OIDC tokens to STS on guardrail auth (#34637) 2026-07-25 16:27:54 -07:00
test_router fix(logging): stop pinning large request payloads past request end (#33455) 2026-07-15 15:28:37 -07:00
types refactor(rate-limits): move the v3 limiter per-request stash off request metadata onto a ContextVar 2026-07-30 14:00:20 -07:00
vector_stores
videos
__init__.py
conftest.py
log.txt
readme.md
test__types.py
test_a2a_registry_lookup.py
test_acompletion_session_reuse_e2e.py
test_add_deployment_no_master_key.py
test_aembedding_session_reuse_e2e.py
test_anthropic_beta_headers_filtering.py
test_anthropic_skills_transformation.py
test_anthropic_sonnet_1hr_cache_pricing.py
test_audio_transcription_rust_bridge.py perf(bedrock): audio transcription via rust core (py->rust bridge) (#33990) 2026-07-20 14:09:41 -07:00
test_azure_ad_token_credential_resolution.py
test_azure_video_router.py
test_batch_completion_models_all_responses.py
test_bedrock_anthropic_1hr_cache_pricing.py
test_bedrock_usgov_haiku_1hr_cache.py
test_bedrock_usgov_pricing.py
test_budget_ratchet_check.py
test_chat_ui_responses_session.py
test_check_licenses.py
test_check_type_discipline.py fix(lint): restrict freezing-wrapper match to bare names and types.MappingProxyType 2026-07-30 22:26:05 -07:00
test_circleci_path_filter.py ci: gate CircleCI jobs on changed paths (#32080) 2026-07-04 19:15:08 -07:00
test_claude_fable_5_config.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_claude_haiku_4_5_config.py
test_claude_opus_4_6_config.py fix(azure_ai): advertise 1M context window for Claude Opus 4.6+ on Foundry 2026-07-24 21:53:12 +00:00
test_claude_opus_4_8_config.py fix(azure_ai): advertise 1M context window for Claude Opus 4.6+ on Foundry 2026-07-24 21:53:12 +00:00
test_claude_opus_5_config.py feat(anthropic): add Claude Opus 5 2026-07-24 10:43:49 -07:00
test_claude_sonnet_4_6_config.py
test_claude_sonnet_5_config.py
test_cloudflare_workers_ai_model_metadata.py
test_command_r7b_pricing.py
test_completion_timeout_resolution.py
test_compression.py
test_constants.py
test_container_router.py
test_cost_calculation_log_level.py
test_cost_calculator.py feat(dashscope): add qwen3.7-plus and qwen3.7-max to the model cost map (#35123) 2026-07-29 14:22:22 -07:00
test_count_tokens_public_api.py
test_dashscope_image_generation.py
test_deepseek_model_metadata.py
test_dockerfile_non_root.py
test_eager_tiktoken_load.py
test_exception_exports.py
test_exception_header_preservation.py
test_exception_mapping_request_attribute.py
test_filter_out_litellm_params.py
test_get_blog_posts.py
test_git_hooks.py
test_github_close_low_quality_prs.py
test_github_review_gate.py
test_github_triage_with_llm.py
test_github_triage_workflows.py
test_gpt_5_4_model_metadata.py fix(pricing): correct gpt-5.4-mini and gpt-5.4-nano token limits 2026-07-30 02:43:54 +00:00
test_gpt_5_5_model_metadata.py
test_gpt_image_cost_calculator.py
test_gpt_realtime_mode.py test(pricing): pin the realtime mode assertion to the bundled cost map (#33806) 2026-07-18 02:52:55 +00:00
test_groq_streaming_encoding.py
test_guardrail_exception_status_codes.py
test_lazy_imports.py
test_litellm_params_reserved_keys.py
test_logging.py
test_lowest_latency_zero_tokens.py
test_main.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_main_module_header.py
test_mistral_medium_3_5_model_metadata.py
test_model_block_unblock.py fix(proxy): stop model writes 500ing on another pod's delete (#35400) 2026-07-31 18:10:48 -07:00
test_model_cost_aliases.py
test_model_param_helper.py
test_model_prices_schema.py ci: enforce format assertions so calendar-impossible deprecation dates fail validation 2026-07-28 16:11:22 -07:00
test_model_response_normalization.py
test_muse_spark_1_1_model_metadata.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_nested_drop_params.py
test_openai_embedding_encoding_format_default.py
test_prisma_generate_if_needed.py
test_project_alias_tracking.py
test_project_tags_pydantic.py
test_rag_openai_ingestion.py
test_rate_limit_error_unification.py test(rate-limits): drop the removed data kwarg from the v3 dynamic limiter raise-branch test 2026-07-30 15:20:44 -07:00
test_redact_string_in_error_paths.py
test_redis.py fix(redis): stop an unreachable Redis from blocking every request (#35273) 2026-07-30 14:36:28 -07:00
test_register_model_custom_pricing.py fix(utils): resolve bedrock regional inference profiles to regional pricing in get_model_info (LIT-4056) (#32389) 2026-07-07 20:49:03 -07:00
test_register_model_zero_cost_persistence.py
test_responses_api_bridge_non_stream.py Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
test_responses_id_security.py fix(responses_id_security): decrypt response ids for input_items follow-ups (#32269) 2026-07-06 14:03:15 -07:00
test_responses_streaming_container_ownership.py
test_retrieve_batch_bedrock_dispatch.py
test_router.py Merge pull request #35028 from BerriAI/litellm_batch_provider_credentials 2026-07-29 11:59:09 -07:00
test_router_block_helpers.py
test_router_exception_redaction.py
test_router_google_genai.py
test_router_model_cost_isolation.py fix(router): propagate capability flags to shared backend cost map key 2026-07-20 16:46:30 -07:00
test_router_order_fallback.py
test_router_per_deployment_num_retries.py fix(router): honor request-level num_retries over global litellm_settings.num_retries (#34124) 2026-07-21 21:32:34 +00:00
test_router_redis_init.py
test_router_retry_non_retryable_errors.py
test_router_retry_policy_update.py
test_router_silent_experiment.py
test_router_streaming_fallback_metadata.py
test_router_weighted_failover.py
test_ruff_strict_gate.py
test_secret_redaction.py fix(key management): enforce minimum custom key length and mask short keys in key_name (#33462) 2026-07-15 18:27:38 -07:00
test_service_logger.py
test_setup_wizard.py
test_shared_session_integration.py
test_ssl_verify_unit.py refactor(bedrock): remove the dead BedrockLLM invoke code path 2026-07-29 20:25:36 -07:00
test_stream_chunk_builder_annotations.py
test_streaming_connection_cleanup.py
test_system_message_format_bug.py
test_thinking_enabled.py
test_triage_rollout_heads_up.py
test_type_check_gate.py
test_type_discipline_gate.py
test_utils.py Merge pull request #35270 from BerriAI/litellm_gpt_pricing_change 2026-07-30 21:46:46 -07:00
test_utils_module_docstring.py
test_uuid_helper.py
test_vcr_safe_body_matcher.py
test_video_generation.py
test_xai_grok_4_3_model_metadata.py
test_xai_responses_auto_routing.py

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py