mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-26 01:12:21 +00:00
05d7fb24bd
* feat(rust): add python-compat crate for Python data formats
Add litellm-python-compat, a PyO3-free crate that reproduces the Python
data formats LiteLLM persists, so Rust readers and writers can interoperate
with state written by the Python proxy:
- literal::literal_eval: a linear recursive-descent port of
ast.literal_eval (prefixes, escapes, implicit concatenation, numeric
underscores and radixes, single unary sign, real +/- complex with 3.14
mixed-mode rules, set(), Python-equality key dedup)
- repr::{repr, to_str}: byte-exact repr()/str(), with a printable table
generated from CPython's str.isprintable (Unicode 16.0.0)
- json::{dumps, from_json, to_json}: json.dumps defaults and the
json.loads mapping
- pickle::{loads, dumps}: plain-data pickles via serde-pickle's serde
interface, which keeps dict insertion order
- truthy::truthy: bool() for plain data
Tests replay fixtures generated by CPython 3.14 (values across every
format and pickle protocol 0-5, plus 154 literal_eval source texts).
Accepted divergences are pinned in a KNOWN table that fails once one
starts matching. A criterion bench covers each format and literal_eval
cost by nesting depth, guarding the linear parse: the py_literal grammar
doubled per nested bracket (105 ms at 16 nested dicts; 19 us at 128 now).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* refactor(rust): split python-compat modules and harden the pickle verifier
- Disable class resolution in scripts/verify_rust_pickles.py, and truncate
the export file once instead of removing and appending to it, so the
verifier cannot be pointed at a pre-created file whose rows execute code
through pickle.loads
- Move Error to error.rs and Value to value.rs, leaving lib.rs as the crate
overview, module list and MAX_DEPTH
- Move the generator and verifier to scripts/, beside the Unicode table
generator, leaving tests/ to the Rust tests
- Group the bench by measured surface, give every case a Throughput so
criterion reports bytes per second, and document baseline comparison
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
23 lines
2.7 KiB
Markdown
23 lines
2.7 KiB
Markdown
- Pure Python *data formats* in Rust, for state Python LiteLLM writes and Rust must read or write byte-compatibly
|
|
- No PyO3, no live objects: truthiness, `__str__`, descriptors of real Python objects belong to `python-bridge`'s coercion layer
|
|
- Format *choices* stay with callers: the `{timestamp, response}` envelope, diskcache modes, and the cache-key recipe live in the cache crates and only call into this crate
|
|
- Intended users
|
|
- `cache-response` codec: reading `str(dict)` values Python's sync Redis path writes (`literal_eval`)
|
|
- `cache-disk`: diskcache's pickled values (`pickle`), falsy-is-miss (`truthy`)
|
|
- Cache-key derivation: sha256 over `str(value)` must match Python byte for byte (`repr::to_str`)
|
|
- Byte-identical writes where Python compares raw values (`json::dumps`, `repr`)
|
|
- Relation to [`py_literal`](https://docs.rs/py_literal/latest/py_literal/): replaced, do not reintroduce
|
|
- Its pest grammar backtracks: parse time doubles per nested `[`/`{` (105 ms at depth 16); ours is linear (19 µs at depth 128)
|
|
- Its formatter is not `repr` (`2e-1`, always single quotes, escapes non-ASCII); it also lost `-0.0`, `(1+2j)`, `set()`
|
|
- `cache-response` and `cache-disk` still depend on it; migrate them here
|
|
- Relation to [`serde-pickle`](https://docs.rs/serde-pickle/latest/serde_pickle/): the pickle codec, used only through its serde interface
|
|
- Never `serde_pickle::Value`: its `BTreeMap` dicts reorder keys
|
|
- Accepted limits: ints beyond i64, `tuple`/`set`/`frozenset` decode as lists, class references (`GLOBAL`/`REDUCE`) fail; writes protocol 3
|
|
- Every behavior is pinned by CPython output, not by reasoning; everything under `generated/` is script output, never hand-edited
|
|
- Regenerate `generated/values.json` with `scripts/generate_fixtures.py`; add a corpus row before changing behavior
|
|
- Divergences go in `KNOWN` in `tests/fixtures.rs` with a reason; an entry that starts matching fails until deleted
|
|
- Regenerate `generated/nonprintable.rs` with `scripts/generate_nonprintable.py` when the target Python's Unicode version changes
|
|
- `scripts/verify_rust_pickles.py` checks CPython reads Rust pickles, with class resolution disabled; CI does not run Python
|
|
- Decoders recurse, so they reject nesting beyond `MAX_DEPTH` for stack safety: deliberately stricter than CPython, whose parser takes ~200 levels and whose unpickler has no limit (pinned as `nested_150`)
|
|
- Formatters (`repr`, `json`) are unbounded; values from the decoders are already capped, a hand-built `Value` is the caller's responsibility
|
|
- `literal_eval` must stay linear in depth: `tests/limits.rs` times the deepest parse, `benches/formats.rs` measures the curve but is manual, since CI runs no Rust bench
|