litellm/tests
ryan-crabbe-berri e889d9d653
fix(bedrock)!: replay redacted_thinking on Converse and keep its data as Claude's own string (#46532)
* fix(bedrock): send redacted_thinking back to Converse as reasoningContent.redactedContent

Replaying an assistant turn that carried a redacted_thinking block to Bedrock
Converse turned it into an empty unsigned reasoningText, which the assistant
content builder then dropped as blank text. Bedrock returned 200 so callers
never noticed. Map redacted_thinking to redactedContent in both the
thinking_blocks and inline content paths of the sync and async builders

* fix(bedrock): drop blank redacted_thinking data and keep non-str signatures signed on Converse replay

* fix(bedrock): replay redacted_thinking to Converse only when the target is Claude

Nova, Llama, DeepSeek and gpt-oss reject reasoningContent.redactedContent with a 400, so a Claude
history replayed to them through a Router fallback or model switch keeps dropping the redacted block
the way main does. Claude is resolved from the Bedrock base model of the target, and an application
inference profile ARN counts as Claude when its deployment's base_model names a Claude model.

Adds integration contracts for the replayed Converse request shape, stream and non-stream, with a
tool call, and for non-Claude and ARN targets.

* test(e2e): cover Bedrock Converse replaying redacted_thinking live through the proxy

Turn 1 triggers redacted thinking on Claude Sonnet 4.6, turn 2 replays it, and a live
edge between the proxy and Bedrock checks the replayed Converse body carries the same
reasoningContent.redactedContent. Covers nonstream, stream and a tool call round trip

* fix(bedrock): validate litellm_params before reading the deployment base_model

litellm_params arrives as an untyped dict, so passing it straight into the typed
helper grew reportUnknownArgumentType. Validating it into Mapping[str, object] first
keeps the read typed without growing the type-check gate

* fix(bedrock): wrap non-Converse redacted_thinking data and let a named model id win over base_model

* test(bedrock): cover redacted_thinking minted outside Converse, model_id and the sync Router base_model path

* fix(bedrock): match bare claude- base_model names without reading the untyped cost map

* fix(bedrock): keep redacted_thinking data as Claude's own string on every route

Converse hands redactedContent back base64-wrapped, and that wrapper used to leak out as data,
so a Converse turn replayed to Invoke or the Anthropic API was rejected. Converse replies now
decode it, Converse requests wrap it exactly once, and the Anthropic-shaped builders unwrap
histories saved with the old wrapping. Responses API encrypted reasoning is no longer sent to
Claude on Converse, which rejected it with a 400
2026-10-14 19:08:14 -07:00
..
_support fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
base_sdk_tests test: move packaging and install checks off CircleCI build_and_test (#45757) 2026-10-10 09:33:51 -07:00
benchmarks refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
code_coverage_tests feat(proxy): general_settings.disable_requester_ip_address_logging keeps caller IP out of spend logs (#45804) 2026-10-14 18:16:40 -07:00
documentation_tests fix(proxy): let configured pass-through endpoints outrank built-in provider prefix routes (#44133) 2026-10-10 12:26:17 -07:00
e2e fix(bedrock)!: replay redacted_thinking on Converse and keep its data as Claude's own string (#46532) 2026-10-14 19:08:14 -07:00
e2e_harness test(e2e): rename load to perf and consolidate the opt-in into E2E_PERF_TESTS (#46077) 2026-10-12 13:54:03 -07:00
fixtures test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
harness_e2e feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
integration fix(bedrock)!: replay redacted_thinking on Converse and keep its data as Claude's own string (#46532) 2026-10-14 19:08:14 -07:00
rust-python-harness docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
test_litellm_rust refactor(rust-bridge): return an unstarted Execution from native entrypoints and drive it in Python (#46299) 2026-10-14 17:50:44 -07:00
typing fix: support OpenAI SDK 3 while retaining SDK 2 compatibility (#44927) 2026-10-09 19:45:50 -07:00
unit fix(bedrock)!: replay redacted_thinking on Converse and keep its data as Claude's own string (#46532) 2026-10-14 19:08:14 -07:00
windows_tests test: move packaging and install checks off CircleCI build_and_test (#45757) 2026-10-10 09:33:51 -07:00
__init__.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py
_master_key.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
_openai_record_replay_proxy.py
_process_helpers.py test: count a zombie grandchild as gone in the migrate deploy timeout test (#42570) 2026-09-22 14:59:26 -07:00
_vcr_conftest_common.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
AGENTS.md docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
capturing_transport.py test(vcr): guard leaked cassette patches and make injected-transport embedding tests immune (#42542) 2026-09-22 14:20:18 -07:00
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD test: finish the non-proxy half of tests/test_litellm (#43281) 2026-09-25 22:43:41 -07:00
test_logging.conf
test_rust_python_harness.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
VCR_TESTING.md test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
white_100x100.png test: stop CI tests from downloading tokenizer files and images (#43257) 2026-09-25 19:27:48 -07:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/unit

This folder can only run mock tests.