litellm/tests
mika-lt c21e576a07
refactor(types): replace Any with proven types in 14 files (#48928)
* refactor(types): replace Any with proven types in 57 files

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(types): hide input values in validation errors and drop secret-bearing seams

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(types): revert the Phoenix prompt seam that rejects string content

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(types): revert files that break mocked tests and Python 3.10 imports

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(types): revert pegasus response validation to main

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(types): revert azure search, milvus and onyx response validation to main

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): keep accepting numeric keys in /cache/delete

* fix(caching): answer 400 for a non-list /cache/delete keys and accept null Stability lists

Return pydantic errors without the caller's input on a bad /cache/delete body, let the Bedrock Stability
image edit schema take null finish_reasons and images like the old reads did, declare user_id and user_role
on the plugin session claim, wrap the retyped Vertex fields in ReadOnly, and drop unused Guardrails AI types

* refactor(types): leave provider reply parsers for a follow-up with live provider calls

Restores the Bedrock Stability, Bedrock KB, AgentCore, Replicate, Recraft, Aleph Alpha and Guardrails AI parsers to main, since no live provider call could prove the stricter schemas. Also types the cache delete route tests.

* refactor(types): keep /cache/delete on main's behavior

Restores the cache delete route and its test to main, so this types PR changes no response. Rejecting a non-list keys is left for its own change, since an object keys deletes its keys on main today.

* fix(types): keep text completion, batch ids and guardrail callback on main's code

* fix(types): keep policy suggestion and auto router parsing on main's code

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-29 07:26:07 -07:00
..
_support fix(proxy)!: end a chat completions stream with an error event when a guardrail blocks mid-stream (#46953) 2026-10-17 18:00:12 -07:00
audio_tests feat(vertex-ai)!: support gemini 3.1 flash tts (#31915) 2026-10-21 04:18:42 +00:00
base_sdk_tests refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
benchmarks test(json_fragment_accumulator): assert peak allocation instead of a drain timing ratio (#43370) 2026-10-28 22:04:47 -07:00
code_coverage_tests feat(responses): support Codex client tool_search on Anthropic and other bridged providers (#48972) 2026-10-29 00:29:40 -07:00
documentation_tests fix(proxy): let configured pass-through endpoints outrank built-in provider prefix routes (#44133) 2026-10-10 12:26:17 -07:00
e2e feat(responses): support Codex client tool_search on Anthropic and other bridged providers (#48972) 2026-10-29 00:29:40 -07:00
e2e_harness test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
fixtures test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
harness_e2e fix(harness): set PWD to the working directory of local sandbox commands (#47530) 2026-10-20 20:06:11 +00:00
integration fix(anthropic)!: drop beta-gated body fields under drop_params (#48793) 2026-10-29 06:25:31 -07:00
rust-python-harness feat(rust): encode LiteLLM response ids on native responses (#46572) 2026-10-18 20:52:34 -07:00
test_litellm_rust fix(anthropic): drop the non-spec custom object from /v1/messages tool definitions (#48893) 2026-10-29 03:36:20 -07:00
typing fix: support OpenAI SDK 3 while retaining SDK 2 compatibility (#44927) 2026-10-09 19:45:50 -07:00
unit refactor(types): replace Any with proven types in 14 files (#48928) 2026-10-29 07:26:07 -07:00
windows_tests refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
__init__.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py
_master_key.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
_openai_record_replay_proxy.py
_process_helpers.py test: count a zombie grandchild as gone in the migrate deploy timeout test (#42570) 2026-09-22 14:59:26 -07:00
_vcr_conftest_common.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
AGENTS.md docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
capturing_transport.py test(vcr): guard leaked cassette patches and make injected-transport embedding tests immune (#42542) 2026-09-22 14:20:18 -07:00
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD test: finish the non-proxy half of tests/test_litellm (#43281) 2026-09-25 22:43:41 -07:00
test_logging.conf
test_rust_python_harness.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
VCR_TESTING.md test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
white_100x100.png test: stop CI tests from downloading tokenizer files and images (#43257) 2026-09-25 19:27:48 -07:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/unit

This folder can only run mock tests.