litellm/tests
tavren27 1046716b6d
fix(anthropic)!: sync the beta header allowlist with what anthropic, vertex_ai, and azure_ai accept (#49409)
* fix(anthropic): sync the beta header allowlist with what anthropic, vertex_ai, and azure_ai accept

* test(e2e): send the azure_ai beta check without the code execution tool

* test(anthropic): require a vertex_ai and azure_ai decision for every anthropic beta

* fix(anthropic): forward the betas Claude Code sends that anthropic, vertex_ai, and azure_ai accept

* test(e2e): send Vertex's header-only betas in the beta acceptance case

* fix(anthropic): keep forwarding Vertex computer-use betas and point beta tests at platforms that still lack them

A live probe found each Claude 4.5 and 4.6 model on Vertex needs one of the two
computer-use betas for the computer tool, so dropping both broke it. The e2e
acceptance test leaves those two out of its all-at-once Vertex call, and the
integration and unit tests that relied on a beta being dropped now use one
the platform still lacks.

* test(anthropic): point unsupported-beta tests at betas the platforms still drop

The Azure AI handler test used claude-code-20250219 as the beta Azure drops, and the
Rust parse test used it as an unknown value. This PR forwards it on Azure AI and maps it,
so both now use values that stay dropped or unknown.

* fix(anthropic): add computer-use-2025-11-24 for the computer_20251124 tool

The computer_20251124 tool fell through to the default computer-use-2024-10-22 beta, which the
provider rejects for that tool version. Map it to computer-use-2025-11-24, which the beta map now
forwards on anthropic and azure_ai

* fix(anthropic): send no computer use beta for an unknown computer tool version

An unknown computer tool version fell back to computer-use-2024-10-22, which the anthropic and azure_ai columns now forward. It now gets no beta, as every platform effectively sent before. Every computer tool version also skips defer_loading and allowed_callers, computer_20251124 included

* test(azure_ai): read the branch's beta map in the auto-injected beta wire test

* fix(anthropic): keep defer_loading and allowed_callers on the computer_20251124 chat tool

* test(anthropic): read the branch's beta map in the azure_ai safeguards beta wire test

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-11-02 05:57:51 -08:00
..
_support refactor(proxy): inject database repositories through per-repository dependencies (#48507) 2026-10-30 22:57:54 +00:00
audio_tests feat(vertex-ai)!: support gemini 3.1 flash tts (#31915) 2026-10-21 04:18:42 +00:00
base_sdk_tests feat(chat): resume replies after browser disconnects (#49461) 2026-10-31 14:43:24 -07:00
benchmarks fix(realtime)!: send the Gemini Live setup at the first content so session.update settings apply (#39864) 2026-10-30 01:31:39 -07:00
code_coverage_tests refactor(router): extract PythonCore behind RouterCore and pass the router its config (#49119) 2026-10-31 12:54:23 -07:00
documentation_tests fix(proxy): let configured pass-through endpoints outrank built-in provider prefix routes (#44133) 2026-10-10 12:26:17 -07:00
e2e fix(anthropic)!: sync the beta header allowlist with what anthropic, vertex_ai, and azure_ai accept (#49409) 2026-11-02 05:57:51 -08:00
e2e_harness refactor(helm)!: connect both charts to a separately deployed Lens (#49169) 2026-10-30 02:29:44 +00:00
fixtures test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
harness_e2e fix(harness): set PWD to the working directory of local sandbox commands (#47530) 2026-10-20 20:06:11 +00:00
integration fix(anthropic)!: sync the beta header allowlist with what anthropic, vertex_ai, and azure_ai accept (#49409) 2026-11-02 05:57:51 -08:00
rust-python-harness feat(rust): encode LiteLLM response ids on native responses (#46572) 2026-10-18 20:52:34 -07:00
test_litellm_rust refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
typing fix: support OpenAI SDK 3 while retaining SDK 2 compatibility (#44927) 2026-10-09 19:45:50 -07:00
unit fix(anthropic)!: sync the beta header allowlist with what anthropic, vertex_ai, and azure_ai accept (#49409) 2026-11-02 05:57:51 -08:00
windows_tests refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
__init__.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py
_master_key.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
_openai_record_replay_proxy.py
_process_helpers.py test: count a zombie grandchild as gone in the migrate deploy timeout test (#42570) 2026-09-22 14:59:26 -07:00
_vcr_conftest_common.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
AGENTS.md docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
capturing_transport.py test(vcr): guard leaked cassette patches and make injected-transport embedding tests immune (#42542) 2026-09-22 14:20:18 -07:00
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD test: finish the non-proxy half of tests/test_litellm (#43281) 2026-09-25 22:43:41 -07:00
test_logging.conf
test_rust_python_harness.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
VCR_TESTING.md test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
white_100x100.png test: stop CI tests from downloading tokenizer files and images (#43257) 2026-09-25 19:27:48 -07:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/unit

This folder can only run mock tests.