* wip
* refactor(rust): share Bedrock paths and Converse/Invoke wire types via llms-types
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust): share Vertex AI locations, base URL and rawPredict paths via llms-types
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust): give every llms-types format and provider its own folder
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust): keep single-adapter projections in llms
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): restructure llms and llms-types AGENTS.md into rules and references
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust): keep provider mod.rs files as re-export entrypoints
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): split llms and llms-types AGENTS.md rules into topic sections
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(rust): move llms-types tests inline and default the workspace to inline tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(rust): move llms tests inline beside the code they cover
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): cover every provider format with AGENTS.md and keep each upstream URL in one file
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* ci: drop the check_rust_agents_md step
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore: regenerate greptile files.json for the new AGENTS.md files
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust): make every llms-types mod.rs a re-export entrypoint and drop check_rust_agents_md
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust): register Messages providers in one match and drop the nested AGENTS.md list
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* docs(agents): ban semicolons and colons in human-facing text
Tighten the writing rule so ";" and ":" are only allowed where the sentence would be nonsensical without them, rewrite the existing semicolons in AGENTS.md to follow it, and note that docs (litellm-docs) keep their trailing "."
* docs(agents): rewrite semicolons in nested AGENTS.md files
* docs(agents): hand-maintain the dashboard Next.js warning and sync e2e CONTRIBUTING lines
* docs(agents): point at the PR template path that exists
* docs(agents): restore the generated Next.js block in the dashboard AGENTS.md
* refactor(rust): derive strum VariantArray and string conversions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): use strum conversions directly with explicit spellings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): use rstest values for key management systems
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(rust): add the Anthropic beta header policy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(rust): resolve known betas held as Other in AnthropicBeta::on
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): use named rstest cases for Other beta resolution
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(rust): make the rstest named-case rule explicit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): move BetaPolicy tests to the public API test crate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): prepare inference and auth foundations
* fix(rust): keep textract operations parsing from kebab-case model names
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Move the generic async coroutine out of the host crate into litellm-coroutine,
with its requirements documented in the crate's AGENTS.md. RouteMachine becomes
CallMachine on top of it, host protocol ops move to host/src/protocol.rs, and
the messages, OCR, host-python, python-bridge and legacy callbacks crates adopt
the new types. Adds error definition rules to litellm-rust/AGENTS.md.
Co-authored-by: Yujong Lee <yujong@berri.ai>
* refactor(rust): move tests.rs files inline or under tests/
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): move tests.rs files inline or under tests/
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): inline path-included test files into their owning src files
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(rust): drop stray proptest regression file
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): cover lowercase, empty and non-authorization headers in bearer detection
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(rust_bridge): count budget-check input tokens in Rust on all LLM routes
Rust counts input tokens from the raw JSON body with the GIL released inside the existing budget reservation, covering every LLM route the auth dependency guards. It only fires for models on the Anthropic tokenizer when a budget is set, and Python counts whenever Rust is off, missing, or declines a body shape.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* perf(rust): count byte-level BPE tokens without the GPT-2 split regex (#40594)
The oniguruma run of the ByteLevel pre-tokenizer regex is about 90% of
encode_fast on a 100k token body (100 ms of the ~110 ms Rust admission
count in the gateway pod). A hand-written scanner that yields the same
pieces, then feeds the model directly, counts the same text in 10 ms.
It only engages for tokenizers with the Anthropic shape (optional NFKC,
ByteLevel without prefix space, no post-processor) and falls back to the
full encoder when the text contains an added token. Parity with
encode_fast is tested on random texts, the pieces are compared with the
real pre-tokenizer, and the \p{L}/\p{N}/\s tables are checked against
oniguruma for every code point.
NFKC runs through unicode-normalization-alignments, the crate and
Unicode tables NormalizedString::nfkc already uses, so the fast path
normalizes exactly what the full encoder would. Using the newer
unicode-normalization crate changed the count for 171 code points that
gained compatibility decompositions after Unicode 9 (U+32FF, U+A7F1..).
The fast normalizer is compared with the tokenizer's for every scalar
value and on random texts.
The scanner is built without mutable state: byte_char and mapped_len replace the const table builders and the reusable mapped buffer, and iter::successors replaces the stateful piece iterator. byte_chars_match_the_byte_level_alphabet checks the byte mapping against ByteLevel for every scalar value.
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(rust_bridge): bound concurrent token-count encodes and share the Anthropic tokenizer predicate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(litellm-rust): move provider transforms into litellm-core + crate allowlist test
* feat(litellm-rust): ai-gateway absorbs route I/O (io/) with lib+server feature split
* refactor(litellm-rust): point python-bridge at litellm-ai-gateway
* build(litellm-rust): macOS pyo3 dynamic_lookup linker flag for cdylib builds
* docs(litellm-rust): 3-crate map in README/AGENTS + refresh CLAUDE boundary
* refactor(litellm-rust): update workspace members to the three crates
