litellm/tests/local_testing
yuneng-jiang 95f33f477d
chore: backport CircleCI speedups (#43347) and cost-map fixes (#42951) to rc/1.104.0 (#43366)
* fix(cost-map): retirement dates, chatgpt reasoning flags, bing pricing, bedrock mantle and mythos, azure gpt-5.6 alias, anthropic batch rates, new nebius, openrouter and xai rows (#42951)

(cherry picked from commit 4179860a17)

* ci: cut CircleCI wall time without loosening test isolation (#43347)

* ci: cut CircleCI wall time without loosening test isolation

* fix(ci): parse integration split files that follow --results

The CircleCI machine image ships Python 3.12.2, whose argparse leaves the
files positional empty when it follows an option and another positional, so
every extensions node exited with 'unrecognized arguments'. Reproduced on
3.12.2; parse_intermixed_args selects the files on 3.12.2, 3.12.13 and 3.13

* test(ci): resolve command references in the Rust toolchain guard

The Windows rustup install moved into the install_windows_toolchain command,
which the guard only recognized for install_rust. It now accepts any command
that installs a pinned rustup and reads the Windows toolchain pin from it

* ci: cache the Windows release cargo build from main

windows_release_wheel rebuilt every dependency with fat LTO on each run. It now
restores the release target and cargo registry saved by main's scheduled run,
drops the workspace crates' fingerprints so they always rebuild from the
checked-out source, and still runs the full LTO link

* ci: run the Windows release wheel build on windows.xlarge

The fat-LTO release build is the slowest job in the pipeline; more cores
speed up the dependency compile ahead of the final link

* ci: skip the Windows fingerprint cleanup when the cargo cache missed

On a cold cache the release fingerprint directory does not exist, and the
CircleCI PowerShell wrapper failed the step on the suppressed not-found error

(cherry picked from commit 635a718ba1)

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-26 16:36:13 -07:00
..
.litellm_cache
auto_router
example_config_yaml
test_configs
azure_fine_tune.jsonl
azure_speech.mp3
batch_job_results_furniture.jsonl
cache_unit_tests.py
conftest.py test: fix stale and state-leaking tests red on scheduled CircleCI (#43266) (#43267) 2026-09-25 19:27:18 -07:00
create_mock_standard_logging_payload.py
data_map.txt
eagle.wav
example.jsonl
gettysburg.wav
large_text.py
model_cost.json
openai_batch_completions.jsonl
openai_batch_completions_router.jsonl
speech_vertex.mp3
stream_chunk_testdata.py
test_acompletion.py
test_acompletion_fallbacks.py
test_acooldowns_router.py
test_add_function_to_prompt.py
test_aim_guardrails.py
test_alangfuse.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_amazing_vertex_completion.py fix: repair seven regressions caught by CircleCI on main (#42640) 2026-09-23 02:26:36 +00:00
test_anthropic_prompt_caching.py
test_arize_ai.py
test_arize_phoenix.py
test_assistants.py
test_async_fn.py
test_auth_utils.py
test_azure_anthropic_sync_post.py
test_azure_openai.py
test_basic_python_version.py fix(ci): let the install smoke test boot its key-less proxy config 2026-09-21 12:56:21 -07:00
test_batch_completion_return_exceptions.py
test_batch_completions.py
test_blocked_user_list.py
test_braintrust.py
test_caching.py fix(redis): authenticate sync clusters with IAM credential providers (#40204) 2026-09-23 17:23:55 -05:00
test_caching_ssl.py
test_completion.py test(ollama): assert the images sent to Ollama instead of echoing them through response (#42905) 2026-09-24 11:54:26 -07:00
test_completion_cost.py test(cost): repoint the Azure image cost test at gpt-image-2 (#42631) 2026-09-22 18:37:41 -07:00
test_completion_with_retries.py
test_config.py
test_cost_calc.py
test_custom_callback_input.py
test_custom_llm.py
test_custom_logger.py
test_disk_cache_unit_tests.py
test_docker_no_network_on_deploy.py
test_dual_cache.py
test_dynamic_rate_limit_handler.py
test_embedding.py test(vcr): guard leaked cassette patches and make injected-transport embedding tests immune (#42542) 2026-09-22 14:20:18 -07:00
test_exceptions.py test: point CircleCI-only suites at models still in the cost map (#42617) 2026-09-22 17:28:34 -07:00
test_fake_openai_endpoint.py
test_file_types.py
test_function_call_parsing.py test: point CircleCI-only suites at models still in the cost map (#42617) 2026-09-22 17:28:34 -07:00
test_function_calling.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_function_setup.py
test_gcs_bucket.py
test_gcs_cache_unit_tests.py
test_gemini_reasoning_content.py
test_get_llm_provider.py test: point CircleCI-only suites at models still in the cost map (#42617) 2026-09-22 17:28:34 -07:00
test_get_model_file.py
test_get_model_info.py chore: backport CircleCI speedups (#43347) and cost-map fixes (#42951) to rc/1.104.0 (#43366) 2026-09-26 16:36:13 -07:00
test_get_optional_params_embeddings.py
test_get_optional_params_functions_not_supported.py
test_google_ai_studio_gemini.py
test_guardrails_ai.py
test_handler_gc_does_not_close_client.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_helicone_integration.py
test_http_parsing_utils.py
test_img_resize.py
test_least_busy_routing.py
test_litellm_max_budget.py
test_llm_guard.py
test_longer_context_fallback.py
test_lowest_cost_routing.py test: point CircleCI-only suites at models still in the cost map (#42617) 2026-09-22 17:28:34 -07:00
test_lowest_latency_routing.py
test_lunary.py
test_mock_request.py
test_model_alias_map.py
test_multiple_deployments.py
test_no_top_level_test_invocations.py
test_ollama.py
test_openai_moderations_hook.py test: point CircleCI-only suites at models still in the cost map (#42617) 2026-09-22 17:28:34 -07:00
test_opik.py
test_pass_through_endpoints.py
test_profiling_router.py
test_prometheus_service.py
test_prompt_injection_detection.py
test_provider_specific_config.py chore(tests): remove commented-out hf, petals and vertex ai completion blocks 2026-09-17 20:03:06 +00:00
test_pydantic.py
test_pydantic_namespaces.py
test_redis_batch_optimizations.py
test_redis_increment_with_floor.py
test_register_model.py test: keep the pinning-test removal free of unrelated reformatting 2026-09-18 04:27:28 +00:00
test_router.py test(bedrock): point unit tests at model ids still in the cost map (#42606) 2026-09-22 16:04:44 -07:00
test_router_batch_completion.py
test_router_budget_limiter.py
test_router_caching.py
test_router_client_init.py
test_router_cooldown_handlers.py test: pin the post-#41289 cooldown contract and scroll the auto-router select spec 2026-09-16 20:52:19 -07:00
test_router_custom_routing.py
test_router_debug_logs.py
test_router_fallback_handlers.py
test_router_fallbacks.py
test_router_get_deployments.py
test_router_max_parallel_requests.py feat(router): reject with 429 when a deployment's max_parallel_requests slots are all in use 2026-09-17 18:14:09 +00:00
test_router_pattern_matching.py
test_router_retries.py
test_router_timeout.py
test_router_utils.py test: point CircleCI-only suites at models still in the cost map (#42617) 2026-09-22 17:28:34 -07:00
test_router_with_fallbacks.py
test_rules.py
test_sagemaker.py
test_sagemaker_nova_integration.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_scheduler.py
test_secret_detect_hook.py
test_spend_calculate_endpoint.py test: point CircleCI-only suites at models still in the cost map (#42617) 2026-09-22 17:28:34 -07:00
test_stream_chunk_builder.py
test_streaming.py test(streaming): remove commented-out retired-provider streaming tests 2026-09-17 20:04:10 +00:00
test_supabase_integration.py
test_team_config.py
test_text_completion.py test: fix five tests left stale by #41311, #41337, #39996 and #41310 2026-09-16 17:47:27 -07:00
test_timeout.py
test_together_ai.py
test_tpm_rpm_routing_v2.py
test_ui_sso_helper_utils.py
test_update_spend.py
test_validate_environment.py
test_wandb.py
user_cost.json
vertex_ai.jsonl
vertex_batch_completions.jsonl
vertex_key.json
whitelisted_bedrock_models.txt fix: repair seven regressions caught by CircleCI on main (#42640) 2026-09-23 02:26:36 +00:00