scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-19 00:01:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 88
litellm/tests/test_litellm
History
yassin fad11fa66e fix(proxy): keep client User-Agent on auth failure spend logs 
Auth gate rejections are raised before add_litellm_data_to_request stamps the caller User-Agent and SpendLogsMetadata dropped the field, so failure spend logs and prometheus labels could not identify an abusive client. Stamp requester_ip_address and user_agent on the failure hook payload and carry user_agent through spend log metadata. Request scopes without a headers entry are tolerated.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 21:02:06 +00:00
..
a2a_protocol fix(proxy): run the remaining inline token counts off the event loop 2026-09-08 18:42:38 +00:00
anthropic_interface refactor(rust): remove per-request enablement arguments (#39928) 2026-09-07 10:43:45 -07:00
batches fix(batches): keep modality token details in raw vertex batch usage 2026-09-15 01:57:50 +00:00
caching Merge pull request #40262 from BerriAI/litellm_lit_7190_inline_token_counts 2026-09-15 13:38:00 -07:00
completion_extras fix(responses bridge): keep mid-conversation system messages in input 2026-09-08 11:44:49 -07:00
compression merge: main into litellm_headroom_protect_cached_prefix 2026-09-15 04:12:28 +00:00
containers
endpoints
enterprise Merge pull request #39836 from BerriAI/litellm_lit_6975_bedrock_files_delete_list 2026-09-12 21:13:27 -07:00
expected_fine_tuning_api
expected_responses_api_request
experimental_mcp_client fix(mcp): bind discovery caching to credentials and active callers 2026-09-11 14:49:00 -07:00
fixtures/together_ai_sync
google_genai
images
integrations Merge pull request #40262 from BerriAI/litellm_lit_7190_inline_token_counts 2026-09-15 13:38:00 -07:00
interactions
litellm_core_utils Merge pull request #40262 from BerriAI/litellm_lit_7190_inline_token_counts 2026-09-15 13:38:00 -07:00
llms Merge remote-tracking branch 'origin/main' into litellm_fix_xai_chat_web_search_live_search 2026-09-15 20:46:51 +00:00
models feat(auto-router): show routed model and savings in Claude Code and Codex (#40330) 2026-09-11 12:52:42 -07:00
ocr Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ocr_custom_pricing 2026-09-12 15:42:59 -07:00
passthrough fix(passthrough): parse Bedrock stream spend incrementally instead of buffering the whole response (#40724) 2026-09-11 09:53:37 -07:00
proxy fix(proxy): keep client User-Agent on auth failure spend logs 2026-09-15 21:02:06 +00:00
rag
realtime_api fix(realtime): send openai and xai health check keys as bearer tokens 2026-09-15 02:20:09 -07:00
repositories feat(auto-router): show routed model and savings in Claude Code and Codex (#40330) 2026-09-11 12:52:42 -07:00
rerank_api
responses Merge remote-tracking branch 'origin/main' into litellm_responses_bridge_filters_unknown_params 2026-09-15 18:15:19 +00:00
router_strategy Merge pull request #41270 from BerriAI/litellm_capability_classifier_pr 2026-09-15 13:04:21 -07:00
router_utils chore: merge main into capability classifier copy 2026-09-15 11:24:01 -07:00
rust_bridge Merge pull request #41180 from BerriAI/litellm_rust_bridge_native_stub 2026-09-15 11:25:56 -07:00
sandbox
secret_managers feat(secret_managers): support customer-managed KMS key for virtual keys stored in AWS Secrets Manager (#40475) 2026-09-11 09:54:08 -07:00
skills feat(skills): semantic search over the LiteLLM-hosted skill registry (#39401) 2026-09-07 12:28:38 -07:00
test_router fix(router): count deployment itpm reservations off the event loop 2026-09-07 21:29:26 -07:00
types Merge pull request #36775 from MvdB/litellm_presidio_new_entities 2026-09-15 11:26:21 -07:00
vector_stores
videos
__init__.py
conftest.py fix(image_edit): await an async transform hook and hide the URL policy verdict from callers 2026-09-04 23:20:21 -07:00
log.txt
readme.md
test_a2a_registry_lookup.py
test_acompletion_session_reuse_e2e.py
test_add_deployment_no_master_key.py
test_aembedding_session_reuse_e2e.py
test_anthropic_beta_headers_filtering.py
test_anthropic_skills_transformation.py
test_assert_ci_coverage.py test: add CircleCI integration contract foundation 2026-09-14 03:30:52 -07:00
test_assert_workflow_dir_hygiene.py
test_audio_transcription_rust_bridge.py refactor(rust): remove per-request enablement arguments (#39928) 2026-09-07 10:43:45 -07:00
test_auto_update_price_and_context_window_file.py feat(cost_map): derive source_revision from the loaded bytes instead of a _metadata stamp 2026-09-07 17:47:51 -07:00
test_azure_ad_token_credential_resolution.py
test_azure_ai_grok_4_3_model_metadata.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_azure_ai_grok_4_6_model_metadata.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_azure_audio_price_aliases.py
test_baseten_glm_5_3_model_metadata.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_batch_completion_models_all_responses.py
test_bedrock_marengo_embed_3_model_metadata.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_bedrock_usgov_pricing.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_budget_ratchet_check.py
test_chat_ui_responses_session.py
test_check_licenses.py
test_check_migrations_no_data_rewrites.py
test_check_py310_typing_imports.py
test_check_test_quality.py
test_check_type_discipline.py
test_circleci_path_filter.py
test_circleci_rust_toolchain.py
test_claude_fable_5_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_claude_haiku_4_5_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_claude_opus_4_6_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_claude_opus_4_8_config.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_claude_opus_5_config.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_claude_sonnet_4_6_config.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_claude_sonnet_5_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_cloudflare_workers_ai_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_command_r7b_pricing.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_completion_timeout_resolution.py
test_component_entrypoint.py feat(proxy): share database connections across workers with an in-container pgbouncer (#39683) 2026-09-10 22:14:37 +00:00
test_compression.py
test_conftest.py
test_conftest_isolation.py
test_constants.py
test_container_router.py
test_cost_calculation_log_level.py
test_cost_calculator.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_cost_map_guard.py feat(cost_map): derive source_revision from the loaded bytes instead of a _metadata stamp 2026-09-07 17:47:51 -07:00
test_count_tokens_public_api.py fix(proxy): run the remaining inline token counts off the event loop 2026-09-08 18:42:38 +00:00
test_dashscope_image_generation.py
test_daybreak_model_metadata.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_deepseek_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_default_branch.py ci: avoid duplicate default branch fetches 2026-09-07 15:28:01 -07:00
test_detect_changes.py
test_dockerfile_apk_repository.py
test_dockerfile_bedrock_realtime_extra.py
test_dockerfile_non_root.py
test_drop_params_env_var.py fix(init): keep non-flag LITELLM_DROP_PARAMS values on with a warning 2026-09-07 22:13:23 -07:00
test_e2e_egress_sentinel.py
test_eager_tiktoken_load.py
test_env_key_doc_gate.py
test_exception_exports.py
test_exception_header_preservation.py fix(bedrock): keep x-amzn-RequestId on chat error responses (#40089) 2026-09-07 17:16:47 -07:00
test_exception_mapping_request_attribute.py
test_filter_out_litellm_params.py
test_fireworks_serverless_model_costs.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_gate_slot_lock.py ci: avoid duplicate default branch fetches 2026-09-07 15:28:01 -07:00
test_gemini_3_1_flash_lite_image_pricing.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_gemini_tts_native_audio_pricing.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_get_blog_posts.py
test_git_hooks.py
test_github_close_low_quality_prs.py
test_github_review_gate.py
test_github_triage_with_llm.py
test_github_triage_workflows.py
test_gpt_5_4_model_metadata.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_gpt_5_5_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_gpt_image_cost_calculator.py
test_gpt_realtime_mode.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_groq_streaming_encoding.py
test_guardrail_exception_status_codes.py
test_lazy_imports.py Revert "perf: lazy-load SDK symbols so import litellm stays under 60 MB RSS (…" 2026-09-05 16:07:09 -07:00
test_lint_workflow_diff_gates.py
test_litellm_params_reserved_keys.py
test_logging.py fix(proxy): read LITELLM_DISABLE_ACCESS_LOG_PATHS per record and match before redaction 2026-09-14 17:27:43 +00:00
test_lowest_latency_zero_tokens.py
test_main.py fix(openai): keep extra_headers out of the chat request body on the httpx handler path 2026-09-14 22:27:35 +00:00
test_main_module_header.py
test_mistral_medium_3_5_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_mistral_small_4_0_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_mistral_zai_glm_5_2_model_metadata.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_model_block_unblock.py
test_model_cost_aliases.py
test_model_param_helper.py
test_model_prices_schema.py feat(pricing): add azure gpt-chat-latest global and data zone rates 2026-09-12 23:34:12 -07:00
test_model_response_normalization.py
test_muse_spark_1_1_model_metadata.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_muse_spark_1_2_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_muse_spark_1_3_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_mutation_report.py
test_nested_drop_params.py
test_non_chat_routes_open_llm_spans.py
test_openai_embedding_encoding_format_default.py
test_openai_service_tier_long_context_pricing.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_pre_commit_lint.py ci: follow the default branch in development tooling 2026-09-07 14:34:45 -07:00
test_prisma_generate_if_needed.py
test_project_alias_tracking.py
test_project_tags_pydantic.py
test_proxy_auth.py
test_rag_openai_ingestion.py
test_rate_limit_error_unification.py
test_redact_string_in_error_paths.py
test_redis.py test(redis): pin ElastiCache IAM signing and TLS coercion invariants 2026-09-10 10:13:03 -04:00
test_redis_credential_provider.py test(redis): pin ElastiCache IAM signing and TLS coercion invariants 2026-09-10 10:13:03 -04:00
test_register_model_custom_pricing.py
test_register_model_zero_cost_persistence.py
test_replicate_model_key_format.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_responses_api_bridge_non_stream.py
test_responses_id_security.py fix(proxy): authorize every Responses API id, not only the ones the proxy issued (#39548) 2026-09-11 11:47:05 -07:00
test_responses_streaming_container_ownership.py
test_retrieve_batch_bedrock_dispatch.py
test_router.py chore: merge main into litellm_lit_1795 and resolve test_router conflict 2026-09-15 19:45:39 +00:00
test_router_block_helpers.py
test_router_exception_redaction.py
test_router_google_genai.py
test_router_model_cost_isolation.py feat(auto-router): allow opted-in team members to manage their routers 2026-09-14 23:56:55 -07:00
test_router_order_fallback.py fix(router): skip the refusing deployment when retrying a non-transient error 2026-09-05 22:25:13 -07:00
test_router_per_deployment_num_retries.py refactor(router): resolve retry policy by exception MRO and add DefaultRetries 2026-09-04 16:09:01 -07:00
test_router_redis_init.py
test_router_retry_backoff_headers.py
test_router_retry_non_retryable_errors.py
test_router_retry_policy_update.py
test_router_silent_experiment.py
test_router_streaming_fallback_metadata.py
test_router_weighted_failover.py fix(router): skip the refusing deployment when retrying a non-transient error 2026-09-05 22:25:13 -07:00
test_ruff_strict_gate.py
test_sambanova_model_metadata.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_secret_redaction.py
test_select_ui_test_scope.py test(ui): align workflow expectations with Vitest 4 2026-09-08 16:13:58 -07:00
test_service_logger.py
test_setup_wizard.py
test_shared_session_integration.py
test_ssl_verify_unit.py
test_stream_chunk_builder_annotations.py
test_stream_chunk_builder_citations.py
test_stream_chunk_builder_images.py
test_streaming_connection_cleanup.py
test_sync_together_ai_models.py feat(cost_map): derive source_revision from the loaded bytes instead of a _metadata stamp 2026-09-07 17:47:51 -07:00
test_system_message_format_bug.py
test_test_quality_gate.py Merge pull request #39870 from BerriAI/litellm_lit_6917_make_check_test_tree_ruff 2026-09-07 10:55:28 -07:00
test_thinking_enabled.py
test_together_ai_model_metadata.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_type_check_gate.py
test_type_discipline_gate.py
test_utils.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_utils_module_docstring.py
test_uuid_helper.py
test_vcr_safe_body_matcher.py
test_video_generation.py fix(cost): carry output_cost_per_second_720p through model info 2026-09-09 14:09:25 +00:00
test_with_dashboard_node.py
test_xai_grok_4_3_model_metadata.py test: drop tests that pin vendor facts and add the CLAUDE.md rule 2026-09-15 18:24:30 +00:00
test_xai_responses_auto_routing.py

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py