litellm/tests
Jesus Velarde 596641454f
Some checks are pending
Cost map pins recheck / rerun-cost-map-guard (push) Waiting to run
LiteLLM Rust DB / db-queries (push) Waiting to run
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
LiteLLM Rust / rust-wheel (push) Waiting to run
Terraform Modules / fmt, validate, test (aws) (push) Waiting to run
Terraform Modules / fmt, validate, test (gcp) (push) Waiting to run
Terraform Provider / gofmt, vet, build, test (push) Waiting to run
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Waiting to run
fix(guardrails): keep BLOCKED verdicts when tool_calls are malformed, ignore non-list tool_calls on /v1/messages
A Generic Guardrail API answer of action BLOCKED with a malformed tool_calls value
raised a validation error before the BLOCKED check, so fail_on_error false let the
request through. tool_calls is now parsed when it is written back, after the verdict.

The /v1/messages output write-back read any returned tool_calls with the right length
as calls, so a guardrail handing back a mapping (HiddenLayer's evaluation dict) failed
the response. It now only reads a list, as the chat handler does.

Also drops the added docstrings and the unrelated test reformatting
2026-11-13 17:42:21 -08:00
..
_support feat(mcp): add an opt-in per-server setting to skip MCP tool output validation (#49961) 2026-11-03 20:42:10 -08:00
audio_tests feat(vertex-ai)!: support gemini 3.1 flash tts (#31915) 2026-10-21 04:18:42 +00:00
base_sdk_tests chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00
basic_proxy_startup_tests feat(docker)!: one shipped image and one entrypoint beside the Dockerfile (#43332) 2026-11-10 18:32:16 -08:00
benchmarks fix(realtime)!: send the Gemini Live setup at the first content so session.update settings apply (#39864) 2026-10-30 01:31:39 -07:00
code_coverage_tests feat(proxy): customer budget rollup, reset on the budget row's window, named templates, team overrides, multi-window limits and pause_allow_increase (#51582) 2026-11-13 16:05:35 +00:00
documentation_tests chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
e2e refactor(redis): delete the Python RedisBatch now that every request op is native (#51383) 2026-11-13 03:22:28 +00:00
e2e_harness test(e2e): add a Codex row where a sub-agent must read the task its parent hands it (#50494) 2026-11-07 21:16:39 +00:00
fixtures test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
harness_e2e fix(harness): set PWD to the working directory of local sandbox commands (#47530) 2026-10-20 20:06:11 +00:00
integration test(otel): keep proxy startup out of the per-test timeout in the excluded services matrix (#51759) 2026-11-14 01:05:05 +00:00
rust-python-harness chore: move workspace packages under litellm (#51000) 2026-11-12 04:19:57 +00:00
test_litellm_rust perf(rust): stop re-serializing the Messages request body in Python logging callbacks (#51440) 2026-11-12 17:13:55 -08:00
typing chore: move config and docs off the repo root (#50966) 2026-11-11 18:29:35 -08:00
unit fix(guardrails): keep BLOCKED verdicts when tool_calls are malformed, ignore non-list tool_calls on /v1/messages 2026-11-13 17:42:21 -08:00
windows_tests feat(proxy): supervise the opt-in Rust forwarding server (#49606) 2026-11-11 19:40:00 +00:00
__init__.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py
_master_key.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
_openai_record_replay_proxy.py
_process_helpers.py
_vcr_conftest_common.py ci: remove CircleCI (#50950) 2026-11-11 15:38:27 -08:00
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
AGENTS.md docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
capturing_transport.py
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD test: finish the non-proxy half of tests/test_litellm (#43281) 2026-09-25 22:43:41 -07:00
test_logging.conf
test_rust_python_harness.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
VCR_TESTING.md test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
white_100x100.png test: stop CI tests from downloading tokenizer files and images (#43257) 2026-09-25 19:27:48 -07:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/unit

This folder can only run mock tests.