litellm/tests/local_testing
Yuneng Jiang e8b9f3675b
test(batch): make the upstream-failure tolerance actually reachable
batch_completion collects per-request failures into its result list rather than
raising them; its own source says "return exceptions if any". So the test's
`except Timeout` and `except litellm.InternalServerError` arms could never fire for
the case they were written for. An upstream 500 instead reached
`response.choices`, raised AttributeError on the exception object, and fell through
to the bare `except Exception` that calls pytest.fail. That is what CircleCI hit.

The tolerance now reads the returned values, which is where the failures actually
are. The same two exception types are tolerated as before, nothing broader.

Checked against four injected outcomes: three InternalServerErrors pass, three
Timeouts pass, an AuthenticationError fails, and a response whose content is None
fails. So it is not tolerating its way to a vacuous green.
2026-08-28 00:25:53 -07:00
..
.litellm_cache
auto_router
example_config_yaml
test_configs
test_model_response_typing
azure_fine_tune.jsonl
azure_speech.mp3
batch_job_results_furniture.jsonl
cache_unit_tests.py
conftest.py
create_mock_standard_logging_payload.py
data_map.txt
eagle.wav
example.jsonl
gettysburg.wav
large_text.py
model_cost.json
openai_batch_completions.jsonl
openai_batch_completions_router.jsonl
speech_vertex.mp3
stream_chunk_testdata.py
test_acompletion.py
test_acompletion_fallbacks.py
test_acooldowns_router.py
test_add_function_to_prompt.py
test_aim_guardrails.py
test_alangfuse.py
test_amazing_vertex_completion.py
test_anthropic_prompt_caching.py
test_arize_ai.py
test_arize_phoenix.py
test_assistants.py
test_async_fn.py
test_auth_utils.py
test_azure_anthropic_sync_post.py
test_azure_openai.py
test_azure_perf.py
test_basic_python_version.py
test_batch_completion_return_exceptions.py
test_batch_completions.py test(batch): make the upstream-failure tolerance actually reachable 2026-08-28 00:25:53 -07:00
test_blocked_user_list.py
test_braintrust.py
test_budget_manager.py
test_cache_preset_key.py
test_caching.py
test_caching_handler.py
test_caching_ssl.py
test_class.py
test_completion.py
test_completion_cost.py
test_completion_with_retries.py
test_config.py
test_cost_calc.py
test_custom_callback_input.py
test_custom_llm.py
test_custom_logger.py
test_disk_cache_unit_tests.py
test_docker_no_network_on_deploy.py
test_dual_cache.py
test_dynamic_rate_limit_handler.py
test_embedding.py
test_exceptions.py
test_fake_openai_endpoint.py
test_file_types.py
test_function_call_parsing.py
test_function_calling.py
test_function_setup.py
test_gcs_bucket.py
test_gcs_cache_unit_tests.py
test_gemini_reasoning_content.py
test_get_llm_provider.py
test_get_model_file.py
test_get_model_info.py
test_get_optional_params_embeddings.py
test_get_optional_params_functions_not_supported.py
test_google_ai_studio_gemini.py
test_guardrails_ai.py
test_helicone_integration.py
test_http_parsing_utils.py
test_img_resize.py
test_langchain_ChatLiteLLM.py
test_least_busy_routing.py
test_litellm_max_budget.py
test_llm_guard.py
test_load_test_router_s3.py
test_loadtest_router.py
test_logging.py
test_longer_context_fallback.py
test_lowest_cost_routing.py
test_lowest_latency_routing.py
test_lunary.py
test_max_tpm_rpm_limiter.py
test_mem_leak.py
test_mem_usage.py
test_mock_request.py
test_model_alias_map.py
test_multiple_deployments.py
test_no_top_level_test_invocations.py
test_ollama.py
test_ollama_local.py
test_ollama_local_chat.py
test_openai_moderations_hook.py
test_opik.py
test_pass_through_endpoints.py
test_profiling_router.py
test_prometheus_service.py
test_prompt_caching.py
test_prompt_injection_detection.py
test_provider_specific_config.py
test_pydantic.py
test_pydantic_namespaces.py
test_redis_batch_optimizations.py
test_register_model.py
test_responses_stream_cache_keys.py
test_router.py
test_router_batch_completion.py
test_router_budget_limiter.py
test_router_caching.py
test_router_client_init.py
test_router_cooldown_handlers.py
test_router_custom_routing.py
test_router_debug_logs.py
test_router_fallback_handlers.py
test_router_fallbacks.py
test_router_get_deployments.py
test_router_max_parallel_requests.py
test_router_pattern_matching.py
test_router_retries.py
test_router_timeout.py
test_router_utils.py
test_router_with_fallbacks.py
test_rules.py
test_sagemaker.py
test_sagemaker_nova_integration.py
test_scheduler.py
test_secret_detect_hook.py
test_spend_calculate_endpoint.py
test_stream_chunk_builder.py
test_streaming.py
test_supabase_integration.py
test_team_config.py
test_text_completion.py
test_timeout.py
test_together_ai.py
test_tpm_rpm_routing_v2.py
test_ui_sso_helper_utils.py
test_unit_test_caching.py
test_update_spend.py
test_validate_environment.py
test_wandb.py
user_cost.json
vertex_ai.jsonl
vertex_batch_completions.jsonl
vertex_key.json
whitelisted_bedrock_models.txt