litellm/tests
devin-ai-integration[bot] 5b042c0db6
fix(rust): add the dangerous-tool-use beta when an anthropic /v1/messages request carries safeguards (#46102)
* fix(rust): add the dangerous-tool-use beta when an anthropic /v1/messages request carries safeguards

* test(rust): check the dangerous-tool-use beta reaches the wire with safeguards

* fix(rust): derive the dangerous-tool-use beta only for hosts with a beta map column

DeepSeek and the other hosts without a column in anthropic_beta_headers_config.json
keep forwarding safeguards and client betas as sent. Adds the wire-level audit tests.

* fix(rust): add the safeguards beta on Azure AI and Claude Platform hosts

Azure AI and Claude Platform Messages transforms now add the
dangerous-tool-use beta for a non-null safeguards field, through the
Azure AI and Anthropic beta-map columns. The safeguards wire and chaos
tests get unique file names so they do not collide with the Python
fix's tests, the Python-route test asserts the fixed behavior behind a
BUG skip, and the DeepSeek test checks that no beta is added

* fix(rust): keep Claude Platform out of the safeguards beta until a route can prove it

Claude Platform has no beta-map column and no QA account, so it goes back to sending
safeguards with no derived beta, like DeepSeek. Also release held chaos-test requests
before the thread pool waits on them, and mark the no-column test as rstest.

* fix(rust): derive the safeguards beta for Claude Platform and keep the null-cell test off Databricks

Claude Platform on AWS answers 400 to safeguards without dangerous-tool-use-2026-09-03 and 200 with it, so its Messages transform derives the beta through the Anthropic column again. The null-cell case now uses only Bedrock Converse, since Databricks support for the beta is changing in a sibling PR.

* fix(rust): drop a null safeguards key instead of sending it without its beta

Anthropic rejects any safeguards key, null included, when the
dangerous-tool-use beta is missing, so an explicit "safeguards": null on the
Rust gateway still answered 400. The shared Messages transform now drops a
null safeguards key on every host, as the proxy already drops None params
before calling Rust, so it answers 200 with no beta.

The scripted peer in the integration wire tests now rejects the key on
presence, matching the real API.

* test: cover Bedrock Mantle in the null safeguards case and match the vendor's non-list wording

The Rust null-safeguards regression test now also runs for bedrock_mantle, whose beta is keyed on the field being present. The scripted Anthropic peer answers a non-list safeguards value with the real API's "valid array" wording.

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-17 16:48:05 +00:00
..
_support fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
base_sdk_tests test: extend runtime regression contracts for Messages and SDK (#46641) 2026-10-15 17:51:15 +00:00
benchmarks fix(anthropic): keep trailing user messages with None or empty list content (#41397) 2026-10-16 07:41:05 +00:00
code_coverage_tests fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
documentation_tests fix(proxy): let configured pass-through endpoints outrank built-in provider prefix routes (#44133) 2026-10-10 12:26:17 -07:00
e2e test(e2e): warm the memory test up with two batches before the measured one (#47087) 2026-10-17 07:09:11 +00:00
e2e_harness test(e2e): event_loop_stalls perf case fails on a gateway event-loop hold over 100 ms that is not on its allowlist (#46974) 2026-10-17 06:55:53 +00:00
fixtures test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
harness_e2e feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
integration fix(rust): add the dangerous-tool-use beta when an anthropic /v1/messages request carries safeguards (#46102) 2026-10-17 16:48:05 +00:00
rust-python-harness docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
test_litellm_rust test(rust-bridge): relocate dispatch tests before routing changes (#46926) 2026-10-16 12:29:33 -07:00
typing fix: support OpenAI SDK 3 while retaining SDK 2 compatibility (#44927) 2026-10-09 19:45:50 -07:00
unit fix(anthropic): add the dangerous-tool-use beta when a /v1/messages request carries safeguards (#43955) 2026-10-17 16:46:25 +00:00
windows_tests test: move packaging and install checks off CircleCI build_and_test (#45757) 2026-10-10 09:33:51 -07:00
__init__.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py
_master_key.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
_openai_record_replay_proxy.py
_process_helpers.py test: count a zombie grandchild as gone in the migrate deploy timeout test (#42570) 2026-09-22 14:59:26 -07:00
_vcr_conftest_common.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
AGENTS.md docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
capturing_transport.py test(vcr): guard leaked cassette patches and make injected-transport embedding tests immune (#42542) 2026-09-22 14:20:18 -07:00
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD test: finish the non-proxy half of tests/test_litellm (#43281) 2026-09-25 22:43:41 -07:00
test_logging.conf
test_rust_python_harness.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
VCR_TESTING.md test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
white_100x100.png test: stop CI tests from downloading tokenizer files and images (#43257) 2026-09-25 19:27:48 -07:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/unit

This folder can only run mock tests.