scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-11 03:38:38 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 83
litellm/tests/test_litellm
History
yassin 2c302d252c test(proxy): reseeded reservation counter also carries the settled request cost 
The reseed test asserted counter == DB floor. The floor is read before the async spend flush writes this request, so the counter now lands at floor plus settled cost, matching the after leg in the PR proof

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-08 21:54:57 +00:00
..
a2a_protocol feat(agentcore-a2a): derive runtime session id from A2A message.contextId (#39371) 2026-09-02 12:40:24 -07:00
anthropic_interface refactor(rust): remove per-request enablement arguments (#39928) 2026-09-07 10:43:45 -07:00
batches Merge pull request #39668 from BerriAI/litellm_lit6899_vertex_batch_tuned_endpoints 2026-09-07 20:28:02 -07:00
caching fix(least-busy): keep the shared count readable, counted once, and off the loop 2026-09-06 00:20:05 -07:00
completion_extras fix(responses bridge): keep mid-conversation system messages in input 2026-09-08 11:44:49 -07:00
compression
containers fix(containers): page upstream until a non-admin container list fills its limit 2026-09-02 21:26:34 -07:00
endpoints fix(speech): honor pcm response_format for Gemini TTS and reject unsupported containers 2026-08-29 16:36:34 -07:00
enterprise fix(ocr): run post-call logging hooks (#40154) 2026-09-08 12:07:25 -07:00
expected_fine_tuning_api
expected_responses_api_request
experimental_mcp_client fix(mcp): strip inbound auth scheme case-insensitively before token exchange (#39346) 2026-09-03 15:08:02 -07:00
fixtures/together_ai_sync
google_genai
images
integrations fix(guardrails): allow framework-supported logging-only mode (#40267) 2026-09-08 12:18:09 -07:00
interactions
litellm_core_utils test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
llms fix(ocr): run post-call logging hooks (#40154) 2026-09-08 12:07:25 -07:00
models fix(proxy): serialize model block responses 2026-08-29 21:22:17 -07:00
ocr test(ocr): add SDK callback E2E parity (#40061) 2026-09-07 11:23:12 -07:00
passthrough fix(passthrough): map sync streaming errors, keep router streaming responses unwrapped, and resolve gigachat from api base 2026-08-31 13:16:40 -07:00
proxy test(proxy): reseeded reservation counter also carries the settled request cost 2026-09-08 21:54:57 +00:00
rag fix(rag): let the managed store's params win over caller kwargs on the search call 2026-09-02 18:59:28 -07:00
realtime_api fix(azure_ai): route audio and realtime calls on Foundry hosts through the Azure OpenAI handlers 2026-09-02 11:05:38 -07:00
repositories fix(router): serialize heuristic tuning quota filters for Prisma 2026-09-07 10:08:19 -07:00
rerank_api fix(rerank): adopt declared authenticating providers in arerank instead of resolving them 2026-09-01 14:47:59 -07:00
responses merge: bring litellm_internal_staging into litellm_fix_agent_mcp_grants 2026-09-07 16:35:40 -07:00
router_strategy chore: merge litellm_internal_staging into litellm_lit_7039_least_busy_shared_counts 2026-09-08 17:11:05 +00:00
router_utils fix(router): give cooldowns their own cache so siblings see a bench in ~1s (#40025) 2026-09-08 10:11:20 -07:00
rust_bridge refactor(rust): remove per-request enablement arguments (#39928) 2026-09-07 10:43:45 -07:00
sandbox
secret_managers fix(azure): restrict the storage credential chain to deployment identities (#39637) 2026-09-03 18:29:32 -07:00
skills feat(skills): semantic search over the LiteLLM-hosted skill registry (#39401) 2026-09-07 12:28:38 -07:00
test_router
types fix(guardrails): accept on_violation block and alert for mcp_security (#40155) 2026-09-07 14:33:02 -07:00
vector_stores Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_python_version_ci 2026-09-03 00:16:19 -07:00
videos
__init__.py
conftest.py fix(image_edit): await an async transform hook and hide the URL policy verdict from callers 2026-09-04 23:20:21 -07:00
log.txt
readme.md
test_a2a_registry_lookup.py
test_acompletion_session_reuse_e2e.py
test_add_deployment_no_master_key.py
test_aembedding_session_reuse_e2e.py
test_anthropic_beta_headers_filtering.py
test_anthropic_skills_transformation.py
test_anthropic_sonnet_1hr_cache_pricing.py
test_assert_ci_coverage.py
test_assert_workflow_dir_hygiene.py
test_audio_transcription_rust_bridge.py refactor(rust): remove per-request enablement arguments (#39928) 2026-09-07 10:43:45 -07:00
test_azure_ad_token_credential_resolution.py
test_azure_ai_grok_4_3_model_metadata.py
test_azure_ai_grok_4_6_model_metadata.py feat(azure_ai): add grok-4.6 to the model cost map 2026-09-02 16:03:45 -07:00
test_azure_audio_price_aliases.py
test_baseten_glm_5_3_model_metadata.py fix(registry): mark baseten GLM-5.3 as vision-capable per Baseten vision docs 2026-09-04 20:10:58 +00:00
test_batch_completion_models_all_responses.py
test_bedrock_anthropic_1hr_cache_pricing.py
test_bedrock_batch_pricing.py
test_bedrock_usgov_pricing.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_budget_ratchet_check.py
test_chat_ui_responses_session.py
test_check_licenses.py
test_check_migrations_no_data_rewrites.py
test_check_py310_typing_imports.py fix: keep litellm importable on Python 3.10 and guard 3.11-only typing imports in CI (#39448) 2026-09-02 18:27:19 -07:00
test_check_test_quality.py
test_check_type_discipline.py
test_circleci_path_filter.py
test_circleci_rust_toolchain.py fix(ci): pin workflow toolchain dependencies 2026-09-02 12:16:25 -07:00
test_claude_fable_5_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_claude_haiku_4_5_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_claude_opus_4_6_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_claude_opus_4_8_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_claude_opus_5_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_claude_sonnet_4_6_config.py
test_claude_sonnet_5_config.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_cloudflare_workers_ai_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_command_r7b_pricing.py
test_completion_timeout_resolution.py
test_component_entrypoint.py fix(docker): match USE_DDTRACE case-insensitively and route build_from_pip through prod_entrypoint.sh (#39344) 2026-09-03 15:10:33 -07:00
test_compression.py
test_conftest.py
test_conftest_isolation.py
test_constants.py
test_container_router.py
test_cost_calculation_log_level.py
test_cost_calculator.py Merge pull request #39850 from BerriAI/litellm_fix_realtime_reasoning_double_bill 2026-09-07 10:55:20 -07:00
test_cost_map_guard.py feat(ci): add the cost map guard check 2026-09-04 17:18:17 -07:00
test_count_tokens_public_api.py
test_dashscope_image_generation.py
test_daybreak_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_deepseek_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_default_branch.py ci: avoid duplicate default branch fetches 2026-09-07 15:28:01 -07:00
test_detect_changes.py
test_dockerfile_apk_repository.py fix(docker): add public Wolfi apk repo to runtime image (#39033) 2026-09-01 15:11:15 -07:00
test_dockerfile_bedrock_realtime_extra.py fix(docker): install bedrock-realtime extra in monolith proxy images (#39223) 2026-09-01 17:51:55 -07:00
test_dockerfile_non_root.py
test_e2e_egress_sentinel.py
test_eager_tiktoken_load.py
test_env_key_doc_gate.py
test_exception_exports.py
test_exception_header_preservation.py fix(bedrock): keep x-amzn-RequestId on chat error responses (#40089) 2026-09-07 17:16:47 -07:00
test_exception_mapping_request_attribute.py
test_filter_out_litellm_params.py
test_fireworks_serverless_model_costs.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_friendli_glm_5_3_flash_model_metadata.py fix(friendli): declare GLM-5.3-Flash reasoning efforts as explicit levels 2026-08-31 13:25:51 -07:00
test_friendli_glm_5_3_model_metadata.py fix(friendli): track GLM-5.3 discounted live pricing and declare effort levels 2026-08-31 13:26:37 -07:00
test_gate_slot_lock.py ci: avoid duplicate default branch fetches 2026-09-07 15:28:01 -07:00
test_gemini_3_1_flash_lite_image_pricing.py
test_gemini_tts_native_audio_pricing.py
test_get_blog_posts.py
test_git_hooks.py
test_github_close_low_quality_prs.py
test_github_review_gate.py
test_github_triage_with_llm.py
test_github_triage_workflows.py
test_gpt_5_4_model_metadata.py
test_gpt_5_5_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_gpt_image_cost_calculator.py fix(registry): drop gpt-image-2 text output price, add openrouter minimax-m3 and qwen3.7-plus 2026-09-04 15:03:29 +00:00
test_gpt_realtime_mode.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_groq_streaming_encoding.py
test_guardrail_exception_status_codes.py
test_lazy_imports.py Revert "perf: lazy-load SDK symbols so import litellm stays under 60 MB RSS (…" 2026-09-05 16:07:09 -07:00
test_litellm_params_reserved_keys.py
test_logging.py fix(logging): redact credential query params from the uvicorn access log (#39293) 2026-09-02 15:10:36 -07:00
test_lowest_latency_zero_tokens.py
test_main.py Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_mistral_voxtral_tts_speech 2026-09-04 16:45:33 -07:00
test_main_module_header.py
test_mistral_medium_3_5_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_mistral_small_4_0_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_mistral_zai_glm_5_2_model_metadata.py
test_model_block_unblock.py
test_model_cost_aliases.py
test_model_param_helper.py
test_model_prices_schema.py fix: declare medium as the only reasoning effort chat-latest accepts 2026-09-05 16:49:00 -07:00
test_model_response_normalization.py
test_muse_spark_1_1_model_metadata.py
test_muse_spark_1_2_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_muse_spark_1_3_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_mutation_report.py
test_nested_drop_params.py
test_non_chat_routes_open_llm_spans.py
test_openai_embedding_encoding_format_default.py
test_openai_service_tier_long_context_pricing.py fix(tests): fold the local price map into the provider model sets 2026-09-03 13:09:46 -07:00
test_pre_commit_lint.py ci: follow the default branch in development tooling 2026-09-07 14:34:45 -07:00
test_prisma_generate_if_needed.py
test_project_alias_tracking.py
test_project_tags_pydantic.py
test_proxy_auth.py
test_rag_openai_ingestion.py
test_rate_limit_error_unification.py
test_redact_string_in_error_paths.py fix(proxy): stop leaking internal exception details to clients (#39380) 2026-09-02 17:32:00 -07:00
test_redis.py fix(redis): coerce env var string types and fix param discovery through decorator wrappers (#30644) 2026-08-31 20:51:31 -07:00
test_register_model_custom_pricing.py fix(cost): bill off-peak rates for deployments that set only off_peak_pricing 2026-09-01 12:14:46 -07:00
test_register_model_zero_cost_persistence.py
test_replicate_model_key_format.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_responses_api_bridge_non_stream.py
test_responses_id_security.py fix(responses): encrypt the response id on every streamed event 2026-09-03 02:41:04 -07:00
test_responses_streaming_container_ownership.py
test_retrieve_batch_bedrock_dispatch.py
test_router.py fix(router): keep provider response headers on streaming chat completions (#40091) 2026-09-07 12:59:57 -07:00
test_router_block_helpers.py
test_router_exception_redaction.py
test_router_google_genai.py
test_router_model_cost_isolation.py fix(cost-map): retry transient boot fetch failures and recover config deployments dropped by a stale cost map (#39230) 2026-09-01 17:49:06 -07:00
test_router_order_fallback.py fix(router): skip the refusing deployment when retrying a non-transient error 2026-09-05 22:25:13 -07:00
test_router_per_deployment_num_retries.py refactor(router): resolve retry policy by exception MRO and add DefaultRetries 2026-09-04 16:09:01 -07:00
test_router_redis_init.py
test_router_retry_backoff_headers.py
test_router_retry_non_retryable_errors.py
test_router_retry_policy_update.py fix: preserve shared optional callbacks 2026-09-02 03:28:04 +00:00
test_router_silent_experiment.py
test_router_streaming_fallback_metadata.py
test_router_weighted_failover.py fix(router): skip the refusing deployment when retrying a non-transient error 2026-09-05 22:25:13 -07:00
test_ruff_strict_gate.py fix: address cross-version CI failures 2026-09-02 14:17:19 -07:00
test_sambanova_model_metadata.py
test_secret_redaction.py fix(proxy): stop leaking internal exception details to clients (#39380) 2026-09-02 17:32:00 -07:00
test_select_ui_test_scope.py
test_service_logger.py
test_setup_wizard.py
test_shared_session_integration.py
test_ssl_verify_unit.py
test_stream_chunk_builder_annotations.py
test_stream_chunk_builder_citations.py
test_stream_chunk_builder_images.py
test_streaming_connection_cleanup.py
test_sync_together_ai_models.py fix(together_ai): stop writing context_length as max_output_tokens in the serverless sync 2026-08-29 15:26:44 -07:00
test_system_message_format_bug.py
test_test_quality_gate.py Merge pull request #39870 from BerriAI/litellm_lit_6917_make_check_test_tree_ruff 2026-09-07 10:55:28 -07:00
test_thinking_enabled.py
test_together_ai_model_metadata.py test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
test_type_check_gate.py
test_type_discipline_gate.py
test_utils.py fix(proxy): load db credentials in the model reconcile so a worker never serves a model before its credential (#39876) 2026-09-08 10:08:24 -07:00
test_utils_module_docstring.py
test_uuid_helper.py
test_vcr_safe_body_matcher.py
test_video_generation.py fix(registry): add vertex veo 3.1 resolution tier pricing per vertex pricing page 2026-09-01 10:15:18 -07:00
test_with_dashboard_node.py
test_xai_grok_4_3_model_metadata.py
test_xai_responses_auto_routing.py

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py