litellm/litellm-rust/crates/python-compat/AGENTS.md at 05d7fb24bd1e9ced3a5877c82e2de00367744037

mirror of https://github.com/BerriAI/litellm.git synced 2026-09-24 00:52:24 +00:00

devin-ai-integration[bot] 05d7fb24bd

feat(rust): add python-compat crate for Python data formats (#42510 )

* feat(rust): add python-compat crate for Python data formats

Add litellm-python-compat, a PyO3-free crate that reproduces the Python
data formats LiteLLM persists, so Rust readers and writers can interoperate
with state written by the Python proxy:

- literal::literal_eval: a linear recursive-descent port of
  ast.literal_eval (prefixes, escapes, implicit concatenation, numeric
  underscores and radixes, single unary sign, real +/- complex with 3.14
  mixed-mode rules, set(), Python-equality key dedup)
- repr::{repr, to_str}: byte-exact repr()/str(), with a printable table
  generated from CPython's str.isprintable (Unicode 16.0.0)
- json::{dumps, from_json, to_json}: json.dumps defaults and the
  json.loads mapping
- pickle::{loads, dumps}: plain-data pickles via serde-pickle's serde
  interface, which keeps dict insertion order
- truthy::truthy: bool() for plain data

Tests replay fixtures generated by CPython 3.14 (values across every
format and pickle protocol 0-5, plus 154 literal_eval source texts).
Accepted divergences are pinned in a KNOWN table that fails once one
starts matching. A criterion bench covers each format and literal_eval
cost by nesting depth, guarding the linear parse: the py_literal grammar
doubled per nested bracket (105 ms at 16 nested dicts; 19 us at 128 now).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(rust): split python-compat modules and harden the pickle verifier

- Disable class resolution in scripts/verify_rust_pickles.py, and truncate
  the export file once instead of removing and appending to it, so the
  verifier cannot be pointed at a pre-created file whose rows execute code
  through pickle.loads
- Move Error to error.rs and Value to value.rs, leaving lib.rs as the crate
  overview, module list and MAX_DEPTH
- Move the generator and verifier to scripts/, beside the Unicode table
  generator, leaving tests/ to the Rust tests
- Group the bench by measured surface, give every case a Throughput so
  criterion reports bytes per second, and document baseline comparison

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

2026-09-22 12:16:56 -07:00

2.7 KiB

Raw Blame History

Pure Python data formats in Rust, for state Python LiteLLM writes and Rust must read or write byte-compatibly
- No PyO3, no live objects: truthiness, __str__, descriptors of real Python objects belong to python-bridge's coercion layer
- Format choices stay with callers: the {timestamp, response} envelope, diskcache modes, and the cache-key recipe live in the cache crates and only call into this crate
Intended users
- cache-response codec: reading str(dict) values Python's sync Redis path writes (literal_eval)
- cache-disk: diskcache's pickled values (pickle), falsy-is-miss (truthy)
- Cache-key derivation: sha256 over str(value) must match Python byte for byte (repr::to_str)
- Byte-identical writes where Python compares raw values (json::dumps, repr)
Relation to py_literal: replaced, do not reintroduce
- Its pest grammar backtracks: parse time doubles per nested [/{ (105 ms at depth 16); ours is linear (19 µs at depth 128)
- Its formatter is not repr (2e-1, always single quotes, escapes non-ASCII); it also lost -0.0, (1+2j), set()
- cache-response and cache-disk still depend on it; migrate them here
Relation to serde-pickle: the pickle codec, used only through its serde interface
- Never serde_pickle::Value: its BTreeMap dicts reorder keys
- Accepted limits: ints beyond i64, tuple/set/frozenset decode as lists, class references (GLOBAL/REDUCE) fail; writes protocol 3
Every behavior is pinned by CPython output, not by reasoning; everything under generated/ is script output, never hand-edited
- Regenerate generated/values.json with scripts/generate_fixtures.py; add a corpus row before changing behavior
- Divergences go in KNOWN in tests/fixtures.rs with a reason; an entry that starts matching fails until deleted
- Regenerate generated/nonprintable.rs with scripts/generate_nonprintable.py when the target Python's Unicode version changes
- scripts/verify_rust_pickles.py checks CPython reads Rust pickles, with class resolution disabled; CI does not run Python
Decoders recurse, so they reject nesting beyond MAX_DEPTH for stack safety: deliberately stricter than CPython, whose parser takes ~200 levels and whose unpickler has no limit (pinned as nested_150)
- Formatters (repr, json) are unbounded; values from the decoders are already capped, a hand-built Value is the caller's responsibility
- literal_eval must stay linear in depth: tests/limits.rs times the deepest parse, benches/formats.rs measures the curve but is manual, since CI runs no Rust bench

2.7 KiB Raw Blame History

2.7 KiB

Raw Blame History