scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-29 06:33:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 200
litellm/tests
History
devin-ai-integration[bot] 3afe779125
fix(cost-map): rolling provider pricing, capability and retirement audit (#48188) 
* fix(cost-map): registry audit 2026-10-24, add provider retirement dates and fix mistral-large-4 and azure_ai sonnet 5.5 prices

Co-authored-by: Hannes Brantner <desktop.stilted-9t@icloud.com>
Co-authored-by: Aasif-Simpplr <aasif.multani@simpplr.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(mistral): derive mistral-large-4 cost expectations from the registry row instead of pinned prices

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(cost-map): add fireworks spot routers and reprice together Qwen3.7-Max

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(cost-map): absorb open registry PRs and update azure_ai mistral-small-2503 retirement date

Co-authored-by: kerry <kerry@berri.ai>

Co-authored-by: Aryan <aryan.m@kelpglobal.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(cost-map): credit absorbed registry PR authors

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Aryan <aryan.m@kelpglobal.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(cost-map): add azure gpt-6.1-terra rows and fireworks models/rune path

Co-authored-by: ceng-p <christianpetersen98@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(cost-map): add azure_ai gpt-6.1-terra global standard row

Co-authored-by: Ahmet Selim FEDAKAR <122402435+selimfedakar@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(cost-map): price mistral-large-4 cache reads at the published $0.14

Co-authored-by: zoveki <349028329+zoveki@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* revert(cost-map): leave mistral-large-4 rows to #48825

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(cost-map): refresh provider pricing and retirement metadata

Absorb the verified GPT-5.6 Terra and Luna price fields from #48709. Add official Groq caching, xAI batch pricing, Moonshot metadata, and retirement dates without replacing existing model entries.

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Hannes Brantner <desktop.stilted-9t@icloud.com>
Co-authored-by: Aasif-Simpplr <aasif.multani@simpplr.com>
Co-authored-by: Aryan <aryan.m@kelpglobal.com>
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: ceng-p <christianpetersen98@gmail.com>
Co-authored-by: Ahmet Selim FEDAKAR <122402435+selimfedakar@users.noreply.github.com>
Co-authored-by: zoveki <349028329+zoveki@users.noreply.github.com>
2026-10-28 16:07:22 +00:00
..
_support fix(proxy)!: end a chat completions stream with an error event when a guardrail blocks mid-stream (#46953) 2026-10-17 18:00:12 -07:00
audio_tests feat(vertex-ai)!: support gemini 3.1 flash tts (#31915) 2026-10-21 04:18:42 +00:00
base_sdk_tests refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
benchmarks refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
code_coverage_tests fix(router): continue Responses mid-stream fallbacks on models without assistant prefill (#47672) 2026-10-28 04:34:40 -07:00
documentation_tests fix(proxy): let configured pass-through endpoints outrank built-in provider prefix routes (#44133) 2026-10-10 12:26:17 -07:00
e2e test: update stale integration and guardrail e2e tests to intended behavior (#48665) 2026-10-28 01:11:25 +00:00
e2e_harness test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758) 2026-10-20 22:31:08 -07:00
fixtures test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
harness_e2e fix(harness): set PWD to the working directory of local sandbox commands (#47530) 2026-10-20 20:06:11 +00:00
integration fix(vertex_ai): send Vertex Claude labels as X-Vertex-AI-Labels header instead of the request body (#48753) 2026-10-28 15:41:50 +00:00
rust-python-harness feat(rust): encode LiteLLM response ids on native responses (#46572) 2026-10-18 20:52:34 -07:00
test_litellm_rust refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages (#48510) 2026-10-27 10:08:25 -07:00
typing fix: support OpenAI SDK 3 while retaining SDK 2 compatibility (#44927) 2026-10-09 19:45:50 -07:00
unit fix(cost-map): rolling provider pricing, capability and retirement audit (#48188) 2026-10-28 16:07:22 +00:00
windows_tests test: move packaging and install checks off CircleCI build_and_test (#45757) 2026-10-10 09:33:51 -07:00
__init__.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py
_master_key.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
_openai_record_replay_proxy.py
_process_helpers.py test: count a zombie grandchild as gone in the migrate deploy timeout test (#42570) 2026-09-22 14:59:26 -07:00
_vcr_conftest_common.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
AGENTS.md docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
capturing_transport.py test(vcr): guard leaked cassette patches and make injected-transport embedding tests immune (#42542) 2026-09-22 14:20:18 -07:00
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD test: finish the non-proxy half of tests/test_litellm (#43281) 2026-09-25 22:43:41 -07:00
test_logging.conf
test_rust_python_harness.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
VCR_TESTING.md test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
white_100x100.png test: stop CI tests from downloading tokenizer files and images (#43257) 2026-09-25 19:27:48 -07:00

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/unit

This folder can only run mock tests.