litellm/tests/unit/rust_bridge
maciej-berri adaa460566
fix(messages): load the anthropic beta map only when a filter has a beta to filter (#52188)
* fix(messages): load the anthropic beta map only when a filter has a beta to filter

Since #52042 the Rust messages route asked Python for the beta map on every
azure_ai, Bedrock Invoke and Foundry call while decoding the request, so the
first such call on each worker downloaded the remote map even when it sent no
anthropic-beta, and the download ran synchronously on the request path.

The bridge now installs the map only once Python has loaded it. When a filter
first meets a beta and no map is installed, it asks the host to load it on the
loader thread and uses the compiled table for that call, as before #52042.

* fix(messages): start at most one background beta map load at a time

A burst of beta-carrying calls while the first download was still in flight
queued one load task per call on the loader thread. A non-blocking lock now
lets only one load wait in the queue, and it is released when that load ends.
2026-11-16 20:03:08 -08:00
..
host fix(messages): load the anthropic beta map only when a filter has a beta to filter (#52188) 2026-11-16 20:03:08 -08:00
messages feat(messages): serve bedrock invoke messages on rust only (#51330) 2026-11-15 16:58:31 -08:00
ocr refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
trace refactor(lens): connect LiteLLM to the independent Lens service (#45529) 2026-10-09 16:17:05 -07:00
__init__.py test: migrate phase 15 legacy tests to tests/unit 2026-09-20 11:55:43 +00:00
AGENTS.md refactor(rust-bridge): decide native routing in the catalog, drop the bypass predicates (#46750) 2026-10-20 15:06:24 -07:00
native_route_wheel_test.py fix(bedrock): omit system prompt from audio transcription and report its token usage (#41565) 2026-10-28 19:23:15 -07:00
stubtest.ini test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_bindings.py refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
test_catalog.py feat(messages): serve bedrock invoke messages on rust only (#51330) 2026-11-15 16:58:31 -08:00
test_clickhouse.py refactor(lens): connect LiteLLM to the independent Lens service (#45529) 2026-10-09 16:17:05 -07:00
test_configuration.py ci(unit): cut unit check time with cheaper per-test resets and fewer non-unit tests (#51864) 2026-11-14 15:41:30 -08:00
test_detectors.py feat(messages): serve anthropic messages on rust only (#51119) 2026-11-12 00:07:20 +00:00
test_execution.py refactor(rust-bridge): return an unstarted Execution from native entrypoints and drive it in Python (#46299) 2026-10-14 17:50:44 -07:00
test_failures.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_fork_guard.py refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
test_logger.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_public_call.py fix(rust): run native async calls through python's async wrapper steps (#50423) 2026-11-10 00:24:20 +00:00
test_runtime.py feat(messages): serve anthropic messages on rust only (#51119) 2026-11-12 00:07:20 +00:00
test_token_counter.py fix(proxy): count the request's tool_choice on the token count routes (#51133) 2026-11-14 07:53:27 -05:00
test_tokenizer.py refactor(sdk): separate core AWS and tokenizer dependencies (#44447) 2026-10-08 19:07:57 -07:00
test_verify_linux_native_wheel.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00