litellm/tests/unit
jesus-berri 3988ce2865
Some checks are pending
smoke / py3.12 (push) Waiting to run
smoke / py3.13 (push) Waiting to run
smoke / py3.14 (push) Waiting to run
smoke / py3.15 (push) Waiting to run
CodSpeed Benchmarks / benchmarks (push) Waiting to run
Cost map pins recheck / rerun-cost-map-guard (push) Waiting to run
Publish lint base counts / publish (basedpyright, litellm/scripts/type_check_gate.py) (push) Waiting to run
Publish lint base counts / publish (openapi-docs, litellm/scripts/openapi_docs_gate.py) (push) Waiting to run
Publish lint base counts / publish (ruff-strict, litellm/scripts/ruff_strict_gate.py) (push) Waiting to run
Publish lint base counts / publish (test-quality, litellm/scripts/test_quality_gate.py) (push) Waiting to run
Publish lint base counts / publish (type-discipline, litellm/scripts/type_discipline_gate.py) (push) Waiting to run
Root entry limit / root-entry-limit (push) Waiting to run
Scorecard supply-chain security / Scorecard analysis (push) Waiting to run
smoke / dashboard-build (push) Waiting to run
smoke / py3.10 (push) Waiting to run
smoke / py3.11 (push) Waiting to run
smoke / all-extras-py3.12 (push) Waiting to run
smoke / all-extras-py3.13 (push) Waiting to run
smoke / windows-sdk (push) Waiting to run
smoke / smoke passed (push) Blocked by required conditions
smoke / proxy-build-from-pip (push) Waiting to run
LiteLLM Rust DB / db-queries (push) Waiting to run
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
LiteLLM Rust / rust-wheel (push) Waiting to run
unit / proxy-infra-db-spend (push) Blocked by required conditions
unit / proxy-infra-root-a-p (push) Blocked by required conditions
unit / proxy-infra-root-q-z (push) Blocked by required conditions
unit / proxy-infra-routing (push) Blocked by required conditions
unit / proxy-subprocess (push) Blocked by required conditions
unit / responses-caching-types (push) Blocked by required conditions
unit / router (push) Blocked by required conditions
unit / rust-bridge-harness (push) Blocked by required conditions
unit / auth-checks (push) Blocked by required conditions
unit / budgets (push) Blocked by required conditions
unit / custom-logging (push) Blocked by required conditions
unit / db-and-spend (push) Blocked by required conditions
unit / endpoints-and-responses (push) Blocked by required conditions
unit / enterprise-routing (push) Blocked by required conditions
unit / guardrails-hooks (push) Blocked by required conditions
unit / jwt-and-keys (push) Blocked by required conditions
unit / key-generation (push) Blocked by required conditions
unit / logging-misc (push) Blocked by required conditions
unit / mcp-oauth (push) Blocked by required conditions
unit / proxy-runtime (push) Blocked by required conditions
unit / proxy-server-core (push) Blocked by required conditions
unit / root (push) Blocked by required conditions
unit / proxy-utils (push) Blocked by required conditions
unit / rust-bridge (push) Waiting to run
unit / assert-shard-coverage (push) Waiting to run
unit / enterprise-managed-files (push) Blocked by required conditions
unit / enterprise-repositories-secrets (push) Blocked by required conditions
unit / proxy-server (push) Blocked by required conditions
unit / lens-python-310 (push) Blocked by required conditions
unit / caching-local (push) Blocked by required conditions
unit / core-utils (push) Blocked by required conditions
unit / endpoints (push) Blocked by required conditions
unit / enterprise-package (push) Blocked by required conditions
unit / integrations (push) Blocked by required conditions
unit / llms-anthropic (push) Blocked by required conditions
unit / llms-bedrock (push) Blocked by required conditions
unit / llms-openai-meta (push) Blocked by required conditions
unit / llms-providers (push) Blocked by required conditions
unit / llms-vertex-ai (push) Blocked by required conditions
unit / mcp-elicitation (push) Blocked by required conditions
unit / mcp-server (push) Blocked by required conditions
unit / proxy-auth (push) Blocked by required conditions
unit / proxy-endpoints (push) Blocked by required conditions
unit / proxy-extras (push) Blocked by required conditions
unit / proxy-feature-endpoints (push) Blocked by required conditions
unit / proxy-hooks-client (push) Blocked by required conditions
unit / ui-unit (push) Waiting to run
unit / docs (push) Waiting to run
unit / helm (push) Waiting to run
unit / OpenAI 2.20.0 compatibility (push) Blocked by required conditions
unit / OpenAI 3.0.0 compatibility (push) Blocked by required conditions
unit / OpenAI 3.25.0 compatibility (push) Blocked by required conditions
unit / redis-py 5.3.1 compatibility (push) Blocked by required conditions
unit / redis-py 6.4.0 compatibility (push) Blocked by required conditions
unit / redis-py 8.0.1 compatibility (push) Blocked by required conditions
unit / coverage (push) Blocked by required conditions
unit / redis-py 7.4.1 compatibility (push) Blocked by required conditions
unit / unit passed (push) Blocked by required conditions
GitHub Actions Security Analysis / zizmor (push) Waiting to run
fix(proxy): honor cancel_on_disconnect on the rerank endpoints (#51252)
* fix(proxy): honor cancel_on_disconnect on the rerank endpoints

* test(proxy): cover a connected rerank succeeding with cancel_on_disconnect on

* test(proxy): cover a connected rerank answer through the cancel watcher in the integration rig

* fix(proxy): read cancel_on_disconnect on rerank the same way the chat path does

* fix(proxy): set up the rerank request logger before routing so a hang-up after dispatch logs the query's tokens like chat

* fix(proxy): keep flag-off rerank logging unchanged and price a cut-off rerank from its query and documents

The rerank endpoint now builds its request logger only under cancel_on_disconnect, so a flag-off rerank logs one unbilled 408 row as on main.

The failure estimate for rerank counts the query and every document sent, prices the deployment the router picked, and bills one query on a model priced per query.

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-11-14 03:59:22 -05:00
..
a2a_protocol refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
accounting feat(proxy): enforce customer org, team and member budgets and limits on the request path (#51489) 2026-11-12 20:43:44 -08:00
anthropic_interface feat(anthropic): forward message threads on native /v1/messages (#50976) 2026-11-10 18:33:54 -08:00
batch_completion test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
batches feat(vertex_ai): label Vertex requests with the caller's key, team, org, and project (#50732) 2026-11-14 00:39:01 -05:00
caching refactor(redis): delete the Python RedisBatch now that every request op is native (#51383) 2026-11-13 03:22:28 +00:00
completion_extras test(logging): scope bridged call recorder to its request (#51339) 2026-11-13 21:34:56 -08:00
compression test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
containers refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
db_scripts chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
decisions feat(nace): add Drex Max decisions and inline images (#48426) 2026-11-12 21:27:37 -08:00
endpoints feat(vertex-ai)!: support gemini 3.1 flash tts (#31915) 2026-10-21 04:18:42 +00:00
enterprise fix(files): match every spelling of a provider file id in the ownership check (#51498) 2026-11-13 05:43:43 +00:00
expected_fine_tuning_api test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
expected_responses_api_request test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
experimental_mcp_client fix(mcp): load deprecated package aliases on demand (#48615) 2026-10-27 19:46:19 +00:00
extraction feat(extraction): add /v1/extraction with Cloudflare Clairic support (#47473) 2026-10-20 15:51:45 -07:00
files fix(hosted_vllm): accept the vLLM production-stack router's file and batch objects (#49068) 2026-10-30 23:11:34 -07:00
fine_tuning fix(fine_tuning): stop forwarding api_version to the OpenAI client (#44995) 2026-10-21 01:58:25 +00:00
fixtures test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
gateway fix(helm): route every gateway allowlist path to the gateway in the componentized ingress (#50760) 2026-11-13 15:06:44 -08:00
google_genai feat(vertex_ai): label Vertex requests with the caller's key, team, org, and project (#50732) 2026-11-14 00:39:01 -05:00
harness fix(streaming)!: send the include_usage chunk with an empty choices array (#47109) 2026-10-21 02:26:05 +00:00
images feat(dashscope): add image edits and text-to-speech (#48998) 2026-10-29 17:48:38 -07:00
integration_support chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
integrations fix(vertex_ai): reuse one Google token across RAG Engine searches and cold bursts (#51239) 2026-11-14 03:12:09 -05:00
interactions fix(interactions): bind Deep Research agents and record their costs (#51517) 2026-11-13 00:20:53 -08:00
litellm_core_utils fix(azure): drop the traceback from stream-cut errors sent to clients and alerts (#51760) 2026-11-14 08:00:58 +00:00
litellm_proxy_extras feat(proxy): customer budget rollup, reset on the budget row's window, named templates, team overrides, multi-window limits and pause_allow_increase (#51582) 2026-11-13 16:05:35 +00:00
llms feat(mistral): add native upstream token counting support (#50652) (#50653) 2026-11-14 03:19:57 -05:00
mcp_client feat(mcp): cache upstream tool lists with an admin-set lifetime (#51160) 2026-11-12 08:48:04 -08:00
models fix(alerts): stop detecting traffic drops (#51338) 2026-11-11 20:22:24 -08:00
ocr test(rust-bridge): relocate dispatch tests before routing changes (#46926) 2026-10-16 12:29:33 -07:00
passthrough fix(bedrock): forward x-pass- headers on the /bedrock/model pass-through routes (#50728) 2026-11-10 21:28:49 +00:00
proxy fix(proxy): honor cancel_on_disconnect on the rerank endpoints (#51252) 2026-11-14 03:59:22 -05:00
rag fix(rag)!: use team provider credentials and allow viewer and JWT team access (#45497) 2026-10-17 23:51:48 +00:00
realtime_api refactor(typing): clear 1141 basedpyright Any errors across 182 files (#50711) 2026-11-11 22:11:47 +00:00
redis fix(proxy): keep a customer budget reset from being undone by a stale hierarchy cache entry (#51694) 2026-11-13 14:21:20 -08:00
repositories fix(proxy): keep carried end-user spend when a budget reset is retried (#51143) 2026-11-14 01:03:19 +00:00
rerank_api test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
responses fix(bedrock): send s3:// image, document and video URLs to Converse as S3 sources (#50577) 2026-11-13 21:33:21 -08:00
router_rust feat(redis): serve the health checks' deployment states from a native store (#50357) 2026-11-13 03:22:28 +00:00
router_strategy fix: preserve routed prompt generation settings (#51702) 2026-11-13 16:00:18 -08:00
router_utils feat(custom-logger-v2): add types, dispatcher and route callback call sites (step 0) (#51214) 2026-11-13 22:08:05 +00:00
rust_bridge refactor(router): count the whole router_usage family on the native store (#50862) 2026-11-13 03:22:28 +00:00
sandbox fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
search feat(search): add Microsoft Web IQ provider (#50883) 2026-11-10 22:43:36 +00:00
secret_managers chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
skills feat(skills): add a litellm-setup agent skill (#47731) 2026-10-28 15:44:43 -07:00
systemone fix(databricks): reuse base OAuth for decisions (#48395) (#48527) 2026-10-26 20:18:16 -07:00
telemetry feat(telemetry): add AggregatingSink, fixed-bucket histograms, Report and HttpExporter (#45487) 2026-10-13 18:33:01 -07:00
test_router feat(custom-logger-v2): add the contract for rust callbacks and bind the proxy request scope (#51526) 2026-11-14 02:33:27 +00:00
text_completions fix(proxy): name the deployment in stream error alerts on chat, completions, responses and messages (#51034) 2026-11-11 22:50:23 +00:00
tracing fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344) 2026-10-16 00:53:52 +00:00
types fix(types): build Usage schemas eagerly so parallel first requests cannot race (#51591) 2026-11-13 12:52:55 -08:00
vector_stores fix(vertex_ai): reuse one Google token across RAG Engine searches and cold bursts (#51239) 2026-11-14 03:12:09 -05:00
videos refactor(typing): clear 1141 basedpyright Any errors across 182 files (#50711) 2026-11-11 22:11:47 +00:00
__init__.py test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
AGENTS.md feat(vector_stores): add OpenSearch k-NN search as a vector store provider (#47759) 2026-10-29 08:15:05 -07:00
conftest.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
create_mock_standard_logging_payload.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_a2a_registry_lookup.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_acompletion_session_reuse_e2e.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_add_deployment_no_master_key.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_aembedding_session_reuse_e2e.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_anthropic_beta_gated_fields.py fix(anthropic)!: drop beta-gated body fields under drop_params (#48793) 2026-10-29 06:25:31 -07:00
test_anthropic_beta_headers_filtering.py fix(bedrock): forward the compact-2026-09-04 compaction param on Invoke /v1/messages (#51223) 2026-11-11 15:10:32 -08:00
test_anthropic_skills_transformation.py feat(anthropic): workload identity federation and pluggable identity sources (#44448) 2026-10-03 17:08:30 -07:00
test_assert_ci_coverage.py ci(unit): split proxy-infra-root in two, isolate subprocess tests, make its file exclusions take effect (#51316) 2026-11-11 17:39:10 -08:00
test_assert_secret_absent.py ci: run the integration groups on GitHub Actions (#48985) 2026-10-29 12:50:08 -07:00
test_assert_workflow_dir_hygiene.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_audio_transcription_rust_bridge.py fix(bedrock): omit system prompt from audio transcription and report its token usage (#41565) 2026-10-28 19:23:15 -07:00
test_azure_ad_token_credential_resolution.py feat(proxy)!: only let clients pick S3 buckets the admin allowed (#47496) 2026-11-06 02:50:15 +00:00
test_azure_ai_grok_4_3_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_azure_ai_grok_4_6_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_baseten_glm_5_3_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_batch_completion_models_all_responses.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_bedrock_marengo_embed_3_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_build_core_distribution.py chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
test_cerebras_llama_model_metadata.py chore(cost-map): remove the 52 rows whose provider retired them before 2026 (#49974) 2026-11-03 16:23:06 -08:00
test_chat_ui_responses_session.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_check_licenses.py feat(proxy): embed enterprise LiteAdmin MCP in LiteLLM images (#44610) 2026-10-05 15:24:34 -07:00
test_check_mcp_operation_boundary.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_check_migrations_lock_timeout.py fix(proxy-extras): bound migration lock waits with SET LOCAL lock_timeout in the migration SQL (#46329) 2026-10-13 20:51:03 -07:00
test_check_migrations_no_data_rewrites.py fix(proxy-extras): build the SpendLogs indexes in the migration job instead of in migrations (#43948) 2026-10-01 14:12:22 -07:00
test_check_openapi_docs.py feat(proxy): customer org, team and member hierarchy with budget templates (#51459) 2026-11-12 15:11:51 -08:00
test_check_py310_typing_imports.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_check_test_quality.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_check_type_discipline.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_check_unbounded_in_lists.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_check_windows_wheel_install.py test: move packaging and install checks off CircleCI build_and_test (#45757) 2026-10-10 09:33:51 -07:00
test_ci_path_filter.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_claude_fable_5_config.py feat: add claude mythos 5.2 to vertex ai cost map (#50940) 2026-11-10 09:35:37 -08:00
test_claude_haiku_5_5_config.py fix(model_prices): consolidate claude-haiku-5-5 over-100k pricing and capability flags (#45151) 2026-10-07 14:14:47 -07:00
test_claude_opus_4_6_config.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_claude_opus_4_8_config.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_claude_opus_5_config.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_claude_sonnet_5_config.py refactor(cost-map): remove openrouter rows, part 5 of 5 (#45669) 2026-10-09 22:48:55 +00:00
test_cli_package.py chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
test_cloudflare_workers_ai_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_completion_timeout_resolution.py fix(router)!: bound fake-streamed calls by the request timeout, not stream_timeout (#47162) 2026-10-17 22:08:02 +00:00
test_compression.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_conftest_isolation.py refactor(model-cost): move the cost map inputs, fetch and register_model overlay into Rust (#50501) 2026-11-10 02:37:35 +00:00
test_constants.py fix(router): cap the default OTPM reservation and reconcile token counters after each call (#49054) 2026-10-29 20:32:04 -07:00
test_container_router.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_cost_calculation_log_level.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_cost_calculator.py fix(openai): add gpt-image-2.6 batch prices and drop gpt-5.5-pro long context batch prices (#51647) 2026-11-13 18:34:38 +00:00
test_cost_map_guard.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_count_tokens_public_api.py feat(mistral): add native upstream token counting support (#50652) (#50653) 2026-11-14 03:19:57 -05:00
test_dashscope_image_generation.py feat(dashscope): add image edits and text-to-speech (#48998) 2026-10-29 17:48:38 -07:00
test_daybreak_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_deepseek_model_metadata.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_default_branch.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_detect_changes.py chore: move ui and vscode-extension under litellm and delete docs (#50992) 2026-11-12 04:19:57 +00:00
test_docker_entrypoint.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_docker_no_network_on_deploy.py chore: move deployment sources under litellm/deploy (#50981) 2026-11-12 04:19:56 +00:00
test_dockerfile_apk_repository.py chore: move deployment sources under litellm/deploy (#50981) 2026-11-12 04:19:56 +00:00
test_dockerfile_bedrock_realtime_extra.py chore: move deployment sources under litellm/deploy (#50981) 2026-11-12 04:19:56 +00:00
test_dockerfile_non_root.py chore: move deployment sources under litellm/deploy (#50981) 2026-11-12 04:19:56 +00:00
test_drop_params_env_var.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_e2e_egress_sentinel.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_eager_tiktoken_load.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_elevenlabs_tts_pricing.py fix(cost-map): rolling registry audit, add elevenlabs api rates, together minimax-m3.1-flash, mistral voxtral 3 and azure_ai phi-4 dates (#47623) 2026-10-23 14:57:35 -07:00
test_empty_mock_response.py fix: honor empty mock completion responses (#48162) (#48266) 2026-10-28 00:56:45 +00:00
test_env_key_doc_gate.py chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
test_exception_exports.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_exception_header_preservation.py fix(openai): keep the upstream error body on e.response for OpenAI and Azure errors (#49079) 2026-10-29 23:30:02 -07:00
test_exception_mapping_request_attribute.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_fake_openai_endpoint.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_filter_out_litellm_params.py fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
test_fireworks_serverless_model_costs.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gate_slot_lock.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_gemini_3_1_flash_lite_image_pricing.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gemini_4_argon_model_metadata.py feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240) 2026-10-15 22:54:23 +00:00
test_gemini_tts_native_audio_pricing.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_get_blog_posts.py chore(blog): refresh the bundled blog posts backup with the latest 50 posts from the docs RSS feed (#49971) 2026-11-03 23:41:15 +00:00
test_git_hooks.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_gpt_5_4_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gpt_5_5_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gpt_image_cost_calculator.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_gpt_realtime_mode.py chore(cost-map): remove the 52 rows whose provider retired them before 2026 (#49974) 2026-11-03 16:23:06 -08:00
test_groq_streaming_encoding.py fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
test_guardrail_exception_status_codes.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_guardrails.py Add guardrails.apply SDK helper (#49217) (#49469) 2026-10-31 23:37:45 +00:00
test_integration_run.py test(integration): isolate every multi-worker integration suite by default (#46027) 2026-10-12 03:46:06 -07:00
test_internal_context.py refactor(redis): delete the Python RedisBatch now that every request op is native (#51383) 2026-11-13 03:22:28 +00:00
test_lazy_imports.py refactor: declare each lazy litellm attribute once and resolve them through one function (#50523) 2026-11-07 09:50:01 -08:00
test_lint_base_counts.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_lint_workflow_diff_gates.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_litellm_cli.py chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
test_litellm_overhead.py test(unit): wait on the work itself instead of fixed sleeps (#46261) 2026-10-13 12:41:06 -07:00
test_litellm_params_reserved_keys.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_logging.py chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
test_lowest_latency_zero_tokens.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_main.py fix(azure_ai): send reasoning_effort to Foundry Kimi K3 and keep its reasoning_content (#51747) 2026-11-14 02:08:11 -05:00
test_main_module_header.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_mistral_large_4_model_metadata.py fix(mistral): declare low, medium and high reasoning effort for mistral-large-4 (#51693) 2026-11-13 12:20:18 -08:00
test_mistral_medium_3_5_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_mistral_small_4_0_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_mistral_zai_glm_5_2_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_model_block_unblock.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_model_cost_aliases.py fix(cost-map): restore backward compatibility after the Rust loader move (#51545) 2026-11-14 08:12:17 +09:00
test_model_param_helper.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_model_prices_and_context_window.py chore(cost-map): add Soniox v6 and date Soniox, Bedrock Claude and Azure Whisper retirements (#51620) 2026-11-13 12:37:12 -08:00
test_model_response_normalization.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_muse_ember_1_0_model_metadata.py feat(cost-map): add meta/muse-ember-1.0 with long-context rates from the Meta pricing page (#50847) 2026-11-10 06:36:06 +00:00
test_muse_spark_1_1_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_muse_spark_1_2_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_muse_spark_1_3_model_metadata.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
test_mutation_report.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_nested_drop_params.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_no_top_level_test_invocations.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_non_chat_routes_open_llm_spans.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_openai_embedding_encoding_format_default.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_openai_record_replay_proxy.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_openai_service_tier_long_context_pricing.py fix(cost_calculator): bill ultrafast prompts above 272k at the ultrafast long-context rates (#43764) 2026-09-30 07:32:16 -07:00
test_openapi_docs_gate.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_package_layout.py test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
test_package_metadata.py fix(proxy): add tzdata to the proxy extra so a legacy TZ doesn't stop startup (#49914) 2026-11-10 23:09:43 -08:00
test_pre_commit_lint.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_prisma_generate_if_needed.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_private_usage_aliases.py chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
test_process_helpers.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_project_alias_tracking.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_project_tags_pydantic.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_proxy_auth.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_pydantic_namespaces.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_rag_openai_ingestion.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_rate_limit_error_unification.py feat(redis): serve the proxy's common Redis call with Rust stores (#50170) 2026-11-06 06:01:11 +00:00
test_read_rc_version.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_recursive_detector.py chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
test_redact_string_in_error_paths.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_redis.py chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
test_redis_credential_provider.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_register_model_custom_pricing.py fix(cost-map): restore backward compatibility after the Rust loader move (#51545) 2026-11-14 08:12:17 +09:00
test_register_model_zero_cost_persistence.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_replicate_model_key_format.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_responses_api_bridge_non_stream.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_responses_id_security.py fix(responses)!: authorize and decode comparison response IDs (#49402) 2026-11-03 03:40:46 +00:00
test_responses_streaming_container_ownership.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_retrieve_batch_bedrock_dispatch.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_router_block_helpers.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_exception_redaction.py fix(streaming)!: map stream read timeouts to litellm.Timeout on chat, messages, and responses (#47198) 2026-10-18 13:58:34 +00:00
test_router_get_settings.py ci(circleci): test Redis behavior against local Redis and print short tracebacks (#44062) 2026-10-01 13:00:54 -07:00
test_router_google_genai.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_router_model_cost_isolation.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_router_order_fallback.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_per_deployment_num_retries.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_redis_init.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_router_retry_backoff_headers.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_retry_non_retryable_errors.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_retry_policy_update.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_silent_experiment.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_streaming_fallback_metadata.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_router_weighted_failover.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_ruff_strict_gate.py chore: move config and docs off the repo root (#50966) 2026-11-11 18:29:35 -08:00
test_sambanova_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_scaledown_model_prices.py feat(scaledown): add native text provider support (#44168) 2026-10-10 14:01:17 -07:00
test_scheduler.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_secret_redaction.py fix(spend-logs): mask token-shaped secrets in stored request and resp… (#48968) 2026-11-12 18:45:06 +00:00
test_seed_tracing_fixtures.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_select_ui_test_scope.py fix(ci): measure PR gates from the target branch's current tip (#49078) 2026-10-29 18:28:16 -07:00
test_service_logger.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_setup_wizard.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_shared_session_integration.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_socket_policy.py test(unit): block external sockets at import time and add a socket policy regression test 2026-09-20 08:07:50 +00:00
test_spend_logs_guard.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_ssl_verify_unit.py test(unit): stop test modules from putting their own directory on sys.path (#43421) 2026-09-28 12:20:20 -07:00
test_stream_chunk_builder_annotations.py fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166) 2026-10-29 08:06:12 -07:00
test_stream_chunk_builder_citations.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_stream_chunk_builder_images.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_streaming_connection_cleanup.py refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
test_system_message_format_bug.py fix(prompt_templates): do not mutate messages in-place in function_call_prompt (#41676) 2026-10-15 14:37:11 -07:00
test_test_quality_gate.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_thinking_enabled.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_together_ai_model_metadata.py chore(cost-map): remove the 52 rows whose provider retired them before 2026 (#49974) 2026-11-03 16:23:06 -08:00
test_tool_loop.py fix(tool_choice): accept the Responses API's flat allowed_tools layout on chat completions (#50467) 2026-11-10 13:04:06 -08:00
test_type_check_gate.py chore: move config and docs off the repo root (#50966) 2026-11-11 18:29:35 -08:00
test_type_discipline_gate.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_typesafe_model_metadata.py fix(cost): price typesafe/jev-1.14.0 (#46594) 2026-10-14 20:27:35 -07:00
test_unit_shard_missing_paths.py ci: render lint, unit and smoke checks as <tier> / <job> with one collector per tier (#45480) 2026-10-09 04:28:41 +00:00
test_unit_shard_per_test_timeout.py ci: render lint, unit and smoke checks as <tier> / <job> with one collector per tier (#45480) 2026-10-09 04:28:41 +00:00
test_utils.py feat(custom-logger-v2): add types, dispatcher and route callback call sites (step 0) (#51214) 2026-11-13 22:08:05 +00:00
test_utils_get_model_info.py chore: move config and docs off the repo root (#50966) 2026-11-11 18:29:35 -08:00
test_utils_get_optional_params.py fix(openai): warn when the GPT-5 reasoning effort gate drops a level under drop_params (#47761) 2026-10-29 04:46:29 -07:00
test_utils_module_docstring.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_uuid_helper.py fix: support Python 3.15 (#48359) 2026-10-26 16:00:11 +00:00
test_vcr_classification.py test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
test_vcr_conftest_common_banner.py ci: remove CircleCI (#50950) 2026-11-11 15:38:27 -08:00
test_vcr_filters.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_vcr_leak_guard.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_vcr_redis_persister.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_vcr_safe_body_matcher.py fix(tests): stop VCR recording and replaying a test's own localhost upstream (#43346) 2026-09-26 15:49:39 -07:00
test_version.py feat(sdk): build core from an independent packaging manifest (#44340) 2026-10-08 18:40:14 -07:00
test_vertex_ai_xai_grok_prompt_caching_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_video_generation.py feat(fal_ai): bill flat per-video fal.ai models with output_cost_per_video (#51225) 2026-11-12 15:13:49 -08:00
test_with_dashboard_node.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
test_ws_vcr.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_xai_grok_4_3_model_metadata.py test: move tests/test_litellm root and small trees into tests/unit (#43186) 2026-09-25 11:30:43 -07:00
test_xai_responses_auto_routing.py fix(router)!: stop a chat-mode Mantle deployment from rewriting the shared Responses routing row (#46263) 2026-10-17 21:39:14 +00:00
whitelisted_bedrock_models.txt chore: move config and docs off the repo root (#50966) 2026-11-11 18:29:35 -08:00