litellm/tests
stimQQ d52ee8f859 feat(apimodels): add apimodels.app chat model prices to the cost map
Adds 36 apimodels/<model> chat rows to model_prices_and_context_window.json and the backup copy, so spend tracking prices apimodels calls instead of logging them at zero

Rates are the per token prices apimodels.app bills, as listed on https://apimodels.app/pricing and on each model page linked in the row's source field. Context limits and the function calling, vision, reasoning and prompt caching flags are copied from the model vendor's own row

The rows carry the same long prompt and off-peak rules the gateway bills with. The gpt-6 and gpt-5.6 rows rebill the whole request at 2x input and cache read and 1.5x output above 272k prompt tokens, claude-haiku-5-5 bills every rate at 5x above 100k, qwen3.7-plus bills 3x above 262,144 tokens through tiered_pricing, and the two DeepSeek Flash rows bill half rate outside 01:00-04:00 and 06:00-10:00 UTC on weekdays

Two cost tests check that an apimodels row bills cached tokens at its cache read rate and that a prompt over 272k tokens moves the whole request to the long context rates
2026-10-19 18:41:09 +09:00
..
_support fix(proxy)!: end a chat completions stream with an error event when a guardrail blocks mid-stream (#46953) 2026-10-17 18:00:12 -07:00
base_sdk_tests test: extend runtime regression contracts for Messages and SDK (#46641) 2026-10-15 17:51:15 +00:00
benchmarks fix(anthropic): keep trailing user messages with None or empty list content (#41397) 2026-10-16 07:41:05 +00:00
code_coverage_tests test(mcp): replace the live Linear MCP OAuth e2e tests with integration coverage (#47154) 2026-10-17 14:29:59 -07:00
documentation_tests fix(proxy): let configured pass-through endpoints outrank built-in provider prefix routes (#44133) 2026-10-10 12:26:17 -07:00
e2e test(perf): align read queries with current dashboard calls (#47243) 2026-10-18 07:00:32 +00:00
e2e_harness test(perf): align read queries with current dashboard calls (#47243) 2026-10-18 07:00:32 +00:00
fixtures test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
harness_e2e feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
integration fix(vertex_ai): price streamed pass-through calls with the regional uplift (#44795) 2026-10-19 08:10:52 +00:00
rust-python-harness feat(rust): encode LiteLLM response ids on native responses (#46572) 2026-10-18 20:52:34 -07:00
test_litellm_rust test(rust): add native vs Python parity harness for responses (#46567) 2026-10-18 20:52:33 -07:00
typing fix: support OpenAI SDK 3 while retaining SDK 2 compatibility (#44927) 2026-10-09 19:45:50 -07:00
unit feat(apimodels): add apimodels.app chat model prices to the cost map 2026-10-19 18:41:09 +09:00
windows_tests test: move packaging and install checks off CircleCI build_and_test (#45757) 2026-10-10 09:33:51 -07:00
__init__.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py
_master_key.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
_openai_record_replay_proxy.py
_process_helpers.py test: count a zombie grandchild as gone in the migrate deploy timeout test (#42570) 2026-09-22 14:59:26 -07:00
_vcr_conftest_common.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
AGENTS.md docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
capturing_transport.py test(vcr): guard leaked cassette patches and make injected-transport embedding tests immune (#42542) 2026-09-22 14:20:18 -07:00
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD test: finish the non-proxy half of tests/test_litellm (#43281) 2026-09-25 22:43:41 -07:00
test_logging.conf
test_rust_python_harness.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
VCR_TESTING.md test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
white_100x100.png test: stop CI tests from downloading tokenizer files and images (#43257) 2026-09-25 19:27:48 -07:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/unit

This folder can only run mock tests.