litellm/tests
islomjon-aiand 707ac1ffee
fix(aiand)!: show ai&'s name and logo, and sync ai&'s model list (#49158)
* feat(ui): show ai&'s name and logo in the dashboard

The Add Model form already lists ai& from its provider metadata, but the dashboard had no entry for it, so the provider showed no logo and model tables printed the raw aiand slug. Adds ai& to the dashboard provider list with its official mark, plated in dark mode so the red and black mark keep their colours, and the aiand/openai/gpt-oss-120b model placeholder

* fix(aiand): retire deepseek-v4-flash, glm-5.2 and kimi-k2.7-code

ai& retired these three models on 2026-10-27 and https://api.aiand.com/v1/models now lists ten. The rows keep their prices with a deprecation_date, so earlier spend still prices, and aiand/* stops listing them, the same way baseten handles retired rows. The other ten rows already match the live prices, context windows and output limits

* feat(aiand): add deepseek-v4.1-flash-1017

ai& launches deepseek-ai/deepseek-v4.1-flash-1017 on 2026-11-02. It is served on https://staging-api.aiand.com/v1/models with the same price, context window, output limit and capabilities as deepseek-v4.1-flash, so its row copies that one

* fix(aiand): retire deepseek-v4.1-flash

ai& retires deepseek-ai/deepseek-v4.1-flash on 2026-10-30, now that deepseek-v4.1-flash-1017 is live in production on https://api.aiand.com/v1/models with the same price, limits and capabilities. Its row keeps its prices with a deprecation_date, like the other three retired ai& rows

* fix(aiand): retire deepseek-v4.1-flash on 2026-11-06

https://api.aiand.com/v1/models now gives deepseek-v4.1-flash a deprecation_date of 2026-11-05T15:00Z, which is 2026-11-06 00:00 in Japan, so its row uses 2026-11-06 instead of 2026-10-30 and aiand/* keeps listing it until then. The wildcard test now leaves a model out only once its retirement date has passed

* test(aiand): derive retired models from the rows instead of pinning dates

The wildcard test now reads each row's deprecation_date, so it holds whichever dates ai& sets, and the test that only restated the JSON dates is gone

* fix(aiand): retire models on their UTC date while the proxy runs, and add muse-spark-open

The wildcard listing now checks each row's deprecation_date when it expands, against the UTC date the field
documents, so a model drops on its day without a restart. deepseek-v4.1-flash now carries 2026-11-05, the
UTC day ai& retires it, and meta/muse-spark-open gets its price row from ai&'s live /v1/models

* fix(aiand): hide retired models only from bundled listings, including the wildcard health check

* fix(aiand): share the UTC retirement check with Together AI and the health check sort

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-30 10:55:26 -07:00
..
_support fix(proxy)!: end a chat completions stream with an error event when a guardrail blocks mid-stream (#46953) 2026-10-17 18:00:12 -07:00
audio_tests feat(vertex-ai)!: support gemini 3.1 flash tts (#31915) 2026-10-21 04:18:42 +00:00
base_sdk_tests build(model-catalog): ship the cost map inside the native extension (#48907) 2026-10-29 07:55:02 -07:00
benchmarks fix(realtime)!: send the Gemini Live setup at the first content so session.update settings apply (#39864) 2026-10-30 01:31:39 -07:00
code_coverage_tests fix(realtime)!: send the Gemini Live setup at the first content so session.update settings apply (#39864) 2026-10-30 01:31:39 -07:00
documentation_tests fix(proxy): let configured pass-through endpoints outrank built-in provider prefix routes (#44133) 2026-10-10 12:26:17 -07:00
e2e fix(realtime)!: send the Gemini Live setup at the first content so session.update settings apply (#39864) 2026-10-30 01:31:39 -07:00
e2e_harness refactor(helm)!: connect both charts to a separately deployed Lens (#49169) 2026-10-30 02:29:44 +00:00
fixtures test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
harness_e2e fix(harness): set PWD to the working directory of local sandbox commands (#47530) 2026-10-20 20:06:11 +00:00
integration fix(proxy): evict a deleted key from every worker's auth cache (#47609) 2026-10-30 10:20:00 -07:00
rust-python-harness feat(rust): encode LiteLLM response ids on native responses (#46572) 2026-10-18 20:52:34 -07:00
test_litellm_rust fix(anthropic): retry /v1/messages without strict when Claude rejects strict tools under drop_params (#49102) 2026-10-29 17:18:59 -07:00
typing fix: support OpenAI SDK 3 while retaining SDK 2 compatibility (#44927) 2026-10-09 19:45:50 -07:00
unit fix(aiand)!: show ai&'s name and logo, and sync ai&'s model list (#49158) 2026-10-30 10:55:26 -07:00
windows_tests refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
__init__.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py
_master_key.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
_openai_record_replay_proxy.py
_process_helpers.py test: count a zombie grandchild as gone in the migrate deploy timeout test (#42570) 2026-09-22 14:59:26 -07:00
_vcr_conftest_common.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
AGENTS.md docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
capturing_transport.py test(vcr): guard leaked cassette patches and make injected-transport embedding tests immune (#42542) 2026-09-22 14:20:18 -07:00
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD test: finish the non-proxy half of tests/test_litellm (#43281) 2026-09-25 22:43:41 -07:00
test_logging.conf
test_rust_python_harness.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
VCR_TESTING.md test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
white_100x100.png test: stop CI tests from downloading tokenizer files and images (#43257) 2026-09-25 19:27:48 -07:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/unit

This folder can only run mock tests.