litellm/tests
jesus-berri 3988ce2865
Some checks are pending
smoke / py3.12 (push) Waiting to run
smoke / py3.13 (push) Waiting to run
smoke / py3.14 (push) Waiting to run
smoke / py3.15 (push) Waiting to run
CodSpeed Benchmarks / benchmarks (push) Waiting to run
Cost map pins recheck / rerun-cost-map-guard (push) Waiting to run
Publish lint base counts / publish (basedpyright, litellm/scripts/type_check_gate.py) (push) Waiting to run
Publish lint base counts / publish (openapi-docs, litellm/scripts/openapi_docs_gate.py) (push) Waiting to run
Publish lint base counts / publish (ruff-strict, litellm/scripts/ruff_strict_gate.py) (push) Waiting to run
Publish lint base counts / publish (test-quality, litellm/scripts/test_quality_gate.py) (push) Waiting to run
Publish lint base counts / publish (type-discipline, litellm/scripts/type_discipline_gate.py) (push) Waiting to run
Root entry limit / root-entry-limit (push) Waiting to run
Scorecard supply-chain security / Scorecard analysis (push) Waiting to run
smoke / dashboard-build (push) Waiting to run
smoke / py3.10 (push) Waiting to run
smoke / py3.11 (push) Waiting to run
smoke / all-extras-py3.12 (push) Waiting to run
smoke / all-extras-py3.13 (push) Waiting to run
smoke / windows-sdk (push) Waiting to run
smoke / smoke passed (push) Blocked by required conditions
smoke / proxy-build-from-pip (push) Waiting to run
LiteLLM Rust DB / db-queries (push) Waiting to run
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
LiteLLM Rust / rust-wheel (push) Waiting to run
unit / proxy-infra-db-spend (push) Blocked by required conditions
unit / proxy-infra-root-a-p (push) Blocked by required conditions
unit / proxy-infra-root-q-z (push) Blocked by required conditions
unit / proxy-infra-routing (push) Blocked by required conditions
unit / proxy-subprocess (push) Blocked by required conditions
unit / responses-caching-types (push) Blocked by required conditions
unit / router (push) Blocked by required conditions
unit / rust-bridge-harness (push) Blocked by required conditions
unit / auth-checks (push) Blocked by required conditions
unit / budgets (push) Blocked by required conditions
unit / custom-logging (push) Blocked by required conditions
unit / db-and-spend (push) Blocked by required conditions
unit / endpoints-and-responses (push) Blocked by required conditions
unit / enterprise-routing (push) Blocked by required conditions
unit / guardrails-hooks (push) Blocked by required conditions
unit / jwt-and-keys (push) Blocked by required conditions
unit / key-generation (push) Blocked by required conditions
unit / logging-misc (push) Blocked by required conditions
unit / mcp-oauth (push) Blocked by required conditions
unit / proxy-runtime (push) Blocked by required conditions
unit / proxy-server-core (push) Blocked by required conditions
unit / root (push) Blocked by required conditions
unit / proxy-utils (push) Blocked by required conditions
unit / rust-bridge (push) Waiting to run
unit / assert-shard-coverage (push) Waiting to run
unit / enterprise-managed-files (push) Blocked by required conditions
unit / enterprise-repositories-secrets (push) Blocked by required conditions
unit / proxy-server (push) Blocked by required conditions
unit / lens-python-310 (push) Blocked by required conditions
unit / caching-local (push) Blocked by required conditions
unit / core-utils (push) Blocked by required conditions
unit / endpoints (push) Blocked by required conditions
unit / enterprise-package (push) Blocked by required conditions
unit / integrations (push) Blocked by required conditions
unit / llms-anthropic (push) Blocked by required conditions
unit / llms-bedrock (push) Blocked by required conditions
unit / llms-openai-meta (push) Blocked by required conditions
unit / llms-providers (push) Blocked by required conditions
unit / llms-vertex-ai (push) Blocked by required conditions
unit / mcp-elicitation (push) Blocked by required conditions
unit / mcp-server (push) Blocked by required conditions
unit / proxy-auth (push) Blocked by required conditions
unit / proxy-endpoints (push) Blocked by required conditions
unit / proxy-extras (push) Blocked by required conditions
unit / proxy-feature-endpoints (push) Blocked by required conditions
unit / proxy-hooks-client (push) Blocked by required conditions
unit / ui-unit (push) Waiting to run
unit / docs (push) Waiting to run
unit / helm (push) Waiting to run
unit / OpenAI 2.20.0 compatibility (push) Blocked by required conditions
unit / OpenAI 3.0.0 compatibility (push) Blocked by required conditions
unit / OpenAI 3.25.0 compatibility (push) Blocked by required conditions
unit / redis-py 5.3.1 compatibility (push) Blocked by required conditions
unit / redis-py 6.4.0 compatibility (push) Blocked by required conditions
unit / redis-py 8.0.1 compatibility (push) Blocked by required conditions
unit / coverage (push) Blocked by required conditions
unit / redis-py 7.4.1 compatibility (push) Blocked by required conditions
unit / unit passed (push) Blocked by required conditions
GitHub Actions Security Analysis / zizmor (push) Waiting to run
fix(proxy): honor cancel_on_disconnect on the rerank endpoints (#51252)
* fix(proxy): honor cancel_on_disconnect on the rerank endpoints

* test(proxy): cover a connected rerank succeeding with cancel_on_disconnect on

* test(proxy): cover a connected rerank answer through the cancel watcher in the integration rig

* fix(proxy): read cancel_on_disconnect on rerank the same way the chat path does

* fix(proxy): set up the rerank request logger before routing so a hang-up after dispatch logs the query's tokens like chat

* fix(proxy): keep flag-off rerank logging unchanged and price a cut-off rerank from its query and documents

The rerank endpoint now builds its request logger only under cancel_on_disconnect, so a flag-off rerank logs one unbilled 408 row as on main.

The failure estimate for rerank counts the query and every document sent, prices the deployment the router picked, and bills one query on a model priced per query.

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-11-14 03:59:22 -05:00
..
_support feat(mcp): add an opt-in per-server setting to skip MCP tool output validation (#49961) 2026-11-03 20:42:10 -08:00
audio_tests feat(vertex-ai)!: support gemini 3.1 flash tts (#31915) 2026-10-21 04:18:42 +00:00
base_sdk_tests chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
basic_proxy_startup_tests feat(docker)!: one shipped image and one entrypoint beside the Dockerfile (#43332) 2026-11-10 18:32:16 -08:00
benchmarks fix(proxy): count tokens with the thinking, effort, and tool_choice that inference sends (#51079) 2026-11-14 02:29:16 +00:00
code_coverage_tests feat(opensearch): map vector store search filters to k-NN filters and add hybrid search (#51131) 2026-11-14 03:30:55 +00:00
documentation_tests chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
e2e refactor(redis): delete the Python RedisBatch now that every request op is native (#51383) 2026-11-13 03:22:28 +00:00
e2e_harness test(e2e): add a Codex row where a sub-agent must read the task its parent hands it (#50494) 2026-11-07 21:16:39 +00:00
fixtures test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
harness_e2e fix(harness): set PWD to the working directory of local sandbox commands (#47530) 2026-10-20 20:06:11 +00:00
integration fix(proxy): honor cancel_on_disconnect on the rerank endpoints (#51252) 2026-11-14 03:59:22 -05:00
rust-python-harness chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
test_litellm_rust perf(rust): stop re-serializing the Messages request body in Python logging callbacks (#51440) 2026-11-12 17:13:55 -08:00
typing chore: move config and docs off the repo root (#50966) 2026-11-11 18:29:35 -08:00
unit fix(proxy): honor cancel_on_disconnect on the rerank endpoints (#51252) 2026-11-14 03:59:22 -05:00
windows_tests feat(proxy): supervise the opt-in Rust forwarding server (#49606) 2026-11-11 19:40:00 +00:00
__init__.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py
_master_key.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
_openai_record_replay_proxy.py
_process_helpers.py
_vcr_conftest_common.py ci: remove CircleCI (#50950) 2026-11-11 15:38:27 -08:00
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
AGENTS.md docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
capturing_transport.py
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD
test_logging.conf
test_rust_python_harness.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
VCR_TESTING.md test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
white_100x100.png

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/unit

This folder can only run mock tests.