mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-17 09:34:48 +00:00
* fix(count_tokens): answer 400, not 500, for a message without role or content Anthropic, Azure AI Anthropic and Bedrock Claude counters now reject a message missing role or content before calling the provider, with the same 400 invalid_request_error the provider answers, instead of a 500 from the count transform. * fix(count_tokens): count a chat message without content the way chat inference sends it, and check every Mantle count * refactor(bedrock): leave message validation to the Bedrock count handlers Main's Bedrock runtime and Mantle count handlers already answer 400 for a message without role or content before any AWS call, so the counter's own check was a second copy * fix(count_tokens): name the chat message chat translation cannot send A contentless system, developer, or tool message, or a message without a role, now answers "Message N must have a 'content' field" or "'role' field" instead of the translator's raw error text, which echoed the whole conversation back * fix(bedrock): keep converse Claude deployments answering 400 for a chat message without content Converse inference drops a non-assistant message without content, and for a last turn Claude then refuses the assistant prefill, so the count has no inference request to match. Invoke and Mantle deployments use Anthropic's chat translation and count the message the way their inference sends it * fix(count_tokens): keep base counting for null-content tool and named turns on Bedrock converse The converse check now covers only messages with just role and content, the ones the merge base rejected. A content null message is never blamed for a translation error, and a body whose messages all drop in translation answers 400 instead of a local estimate * fix(count_tokens): reject only contentless user turns before counting on Bedrock converse A system turn with content null counted on the merge base and converse inference serves it, so it goes through chat translation again * fix(count_tokens): keep the merge base's converse handling and leave null turns chat translation cannot send as given * fix(count_tokens): keep a non-string message role from crashing the Anthropic count checks * test(integration): expect the 400 for a count whose leading system message has no content --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _support | ||
| audio_tests | ||
| base_sdk_tests | ||
| basic_proxy_startup_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| e2e_harness | ||
| fixtures | ||
| harness_e2e | ||
| integration | ||
| rust-python-harness | ||
| test_litellm_rust | ||
| typing | ||
| unit | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _master_key.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_logging.conf | ||
| test_rust_python_harness.py | ||
| VCR_TESTING.md | ||
| white_100x100.png | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/unit
This folder can only run mock tests.