scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-14 23:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 156
litellm/tests/local_testing
History
Exact
Yuneng Jiang b6557d2b14
test: repair three failing suites on litellm_internal_staging 
The management route-coverage guard fires because /team/metadata_schema landed
in #33353 without a behavior-suite scenario, so this adds one covering the nine
seeded actors plus the unauthenticated 401

The prometheus budget-metric assertions read the log call's first positional
arg, which #35703 turned into an unrendered "%s" format string when it moved
logging to lazy args. They now render the message from the call args, which
also pins the arg order and the exception text that the old substring check
never reached

GitHub Models was fully retired on 2026-07-30, so test_completion_github_api
can no longer pass: the endpoint the github provider targets returns 404 and
models.github.ai answers 410 "github_models_retirement_brownout". The dead live
test is removed rather than skipped
2026-08-04 16:07:02 -07:00
..
.litellm_cache
auto_router
example_config_yaml test: test 2026-03-28 19:17:38 -07:00
test_configs test: test 2026-03-28 19:17:38 -07:00
test_model_response_typing
azure_fine_tune.jsonl
azure_speech.mp3
batch_job_results_furniture.jsonl
cache_unit_tests.py
conftest.py feat: declarative fallback generalizations for unknown models (#29718) 2026-06-27 21:01:19 -07:00
create_mock_standard_logging_payload.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
data_map.txt
eagle.wav
example.jsonl
gettysburg.wav
large_text.py
model_cost.json
openai_batch_completions.jsonl
openai_batch_completions_router.jsonl
speech_vertex.mp3
stream_chunk_testdata.py
test_acompletion.py
test_acompletion_fallbacks.py
test_acooldowns_router.py test: test 2026-03-28 19:17:38 -07:00
test_add_function_to_prompt.py
test_add_update_models.py
test_aim_guardrails.py fix(guardrails): return 400 not 500 when AIM blocks a request (#30573) 2026-06-16 18:56:14 -07:00
test_alangfuse.py test: update key names 2026-03-28 21:13:16 -07:00
test_amazing_vertex_completion.py test(vertex_ai): bump local_testing vertex tests from gemini-2.5-flash to gemini-3.5-flash (#32439) 2026-07-08 00:17:55 -07:00
test_anthropic_prompt_caching.py fix(anthropic): self-heal on missing thinking-signature errors from Bedrock/Vertex (#33719) 2026-07-17 18:18:38 +00:00
test_arize_ai.py test: rename env var 2026-03-28 20:27:39 -07:00
test_arize_phoenix.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_assistants.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
test_async_fn.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_auth_utils.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_azure_anthropic_sync_post.py fix(mcp): honor server_id for REST tool calls with shared upstream URLs (#30184) 2026-06-12 07:25:53 -07:00
test_azure_content_safety.py
test_azure_openai.py test: test 2026-03-28 19:17:38 -07:00
test_azure_perf.py test: test 2026-03-28 19:17:38 -07:00
test_basic_python_version.py fix(deps): raise aiohttp floor to 3.14.2 to clear pooled-connection timeouts 2026-07-31 00:08:20 -07:00
test_batch_completion_return_exceptions.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_batch_completions.py replace retired claude-3-haiku-20240307 with claude-haiku-4-5-20251001 in local_testing part1 and router fallback tests 2026-04-20 16:10:45 -07:00
test_blocked_user_list.py
test_braintrust.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_budget_manager.py
test_cache_preset_key.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_caching.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_caching_handler.py fix(caching): replay openai/responses bridge cache hits as chat streams (#28158) 2026-05-18 16:27:06 -07:00
test_caching_ssl.py Merge main and resolve conflict in test_router_client_init.py 2026-03-30 18:44:33 -07:00
test_class.py test: test 2026-03-28 19:17:38 -07:00
test_completion.py test: repair three failing suites on litellm_internal_staging 2026-08-04 16:07:02 -07:00
test_completion_cost.py test(fireworks): mock remaining live smoke tests 2026-05-15 22:28:27 -07:00
test_completion_with_retries.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_config.py test(proxy): assert _delete_deployment's still-desired id set instead of a delete count 2026-08-01 14:36:30 -07:00
test_cost_calc.py fix(main): stop per-request custom pricing from clobbering shared model_cost pricing (#32163) 2026-07-07 10:25:31 -07:00
test_custom_api_logger.py
test_custom_callback_input.py fix(tests): replace shut-down gpt-4o-audio-preview with gpt-audio-1.5 (#28281) 2026-05-19 14:48:30 -07:00
test_custom_llm.py Litellm oss staging 040626 (#29671) 2026-06-04 11:07:20 -07:00
test_custom_logger.py Mark test_redis_cache_completion_stream as flaky with retries 2026-03-15 20:44:18 -07:00
test_disk_cache_unit_tests.py
test_docker_no_network_on_deploy.py build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
test_dual_cache.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_dynamic_rate_limit_handler.py fix(dynamic_rate_limiter): inject clock so active-project window is stable within a request (#32299) 2026-07-06 18:12:47 -07:00
test_dynamodb_logs.py
test_embedding.py fix(bedrock): drop strict/additionalProperties from toolSpec for Claude Sonnet 4 (#31943) 2026-07-01 23:56:25 -07:00
test_exceptions.py fix: cleanup tests 2026-03-30 16:24:35 -07:00
test_fake_openai_endpoint.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
test_file_types.py
test_function_call_parsing.py Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
test_function_calling.py Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
test_function_setup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_gcs_bucket.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_gcs_cache_unit_tests.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_gemini_reasoning_content.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_get_llm_provider.py refactor(fallback-generalizations): split rules into routing and provider-neutral capability kinds 2026-07-11 00:27:45 -07:00
test_get_model_file.py Revert "Merge pull request #16590 from Chesars/refactor/remove-backup-file-dry-principle" 2026-04-25 17:10:41 -03:00
test_get_model_info.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_get_optional_params_embeddings.py Litellm oss staging (#29492) 2026-06-02 08:48:10 -07:00
test_get_optional_params_functions_not_supported.py
test_google_ai_studio_gemini.py
test_guardrails_ai.py
test_helicone_integration.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_http_parsing_utils.py
test_img_resize.py
test_lakera_ai_prompt_injection.py
test_langchain_ChatLiteLLM.py
test_langsmith.py
test_least_busy_routing.py
test_litellm_max_budget.py
test_llm_guard.py fix(llm_guard): apply sanitized prompt returned by moderation API to request (#33331) 2026-07-16 01:27:44 +03:00
test_load_test_router_s3.py
test_loadtest_router.py test: test 2026-03-28 19:17:38 -07:00
test_logfire.py
test_logging.py
test_longer_context_fallback.py
test_lowest_cost_routing.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_lowest_latency_routing.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
test_lunary.py fix(tests): drop module-level test calls that break local_testing collection (#29520) 2026-06-02 13:07:05 -07:00
test_max_tpm_rpm_limiter.py
test_mem_leak.py
test_mem_usage.py
test_mock_request.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_model_alias_map.py fix(test): scope ERROR log assertion to LiteLLM logger in test_model_alias_map 2026-04-29 03:48:41 +00:00
test_model_max_token_adjust.py
test_multiple_deployments.py fix(tests): drop module-level test calls that break local_testing collection (#29520) 2026-06-02 13:07:05 -07:00
test_no_top_level_test_invocations.py fix(tests): drop module-level test calls that break local_testing collection (#29520) 2026-06-02 13:07:05 -07:00
test_ollama.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_ollama_local.py
test_ollama_local_chat.py
test_openai_moderations_hook.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_opik.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_pass_through_endpoints.py fix(passthrough): stream non-sse passthrough responses instead of buffering in memory (#32386) 2026-07-07 20:51:15 -07:00
test_profiling_router.py
test_prometheus_service.py
test_prompt_caching.py
test_prompt_injection_detection.py test: test 2026-03-28 19:17:38 -07:00
test_promptlayer_integration.py
test_provider_specific_config.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_pydantic.py
test_pydantic_namespaces.py
test_redis_batch_optimizations.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_register_model.py fix(tests): drop import-time completion call in test_register_model (#29521) 2026-06-02 16:10:43 -07:00
test_responses_stream_cache_keys.py fix(cache): persist and replay streamed Responses API requests (#24580) 2026-05-01 11:55:36 +05:30
test_router.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
test_router_auto_router.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_batch_completion.py
test_router_budget_limiter.py test: test 2026-03-28 19:17:38 -07:00
test_router_caching.py test: test 2026-03-28 19:17:38 -07:00
test_router_client_init.py test_router_init_azure_service_principal_with_secret_with_environment_variables 2026-03-30 21:15:53 -07:00
test_router_cooldown_handlers.py test: test 2026-03-28 19:17:38 -07:00
test_router_custom_routing.py chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
test_router_debug_logs.py feat(cli): per-agent lite claude / codex / opencode commands that wrap coding agents through the proxy (#29850) 2026-06-10 13:52:26 -07:00
test_router_fallback_handlers.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
test_router_fallbacks.py fix(main): stop per-request custom pricing from clobbering shared model_cost pricing (#32163) 2026-07-07 10:25:31 -07:00
test_router_get_deployments.py
test_router_max_parallel_requests.py fix(tests/vcr): make Redis cassette cache replay deterministically (zero VCR misses on consecutive runs) (#28826) 2026-05-26 11:30:44 -07:00
test_router_pattern_matching.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_retries.py
test_router_timeout.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_router_utils.py test: test 2026-03-28 19:17:38 -07:00
test_router_with_fallbacks.py
test_rules.py
test_sagemaker.py Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
test_sagemaker_nova_integration.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_scheduler.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_secret_detect_hook.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
test_spend_calculate_endpoint.py
test_stream_chunk_builder.py fix(tests): replace shut-down gpt-4o-audio-preview with gpt-audio-1.5 (#28281) 2026-05-19 14:48:30 -07:00
test_streaming.py Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
test_supabase_integration.py
test_team_config.py
test_text_completion.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_timeout.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_together_ai.py
test_tpm_rpm_routing_v2.py fix: drain logging worker in test_router_caching_ttl to remove flake 2026-04-23 14:48:02 -07:00
test_traceloop.py
test_ui_sso_helper_utils.py
test_unit_test_caching.py style: black format test_unit_test_caching.py 2026-04-15 18:19:04 -07:00
test_update_spend.py
test_validate_environment.py
test_wandb.py fix(tests): drop module-level test calls that break local_testing collection (#29520) 2026-06-02 13:07:05 -07:00
user_cost.json
vertex_ai.jsonl
vertex_batch_completions.jsonl
vertex_key.json test: update to new vertex ai keys 2026-03-28 20:19:05 -07:00
whitelisted_bedrock_models.txt Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00