scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-03 02:22:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 81
litellm/litellm-rust/crates/core/AGENTS.md
Yujong Lee 5a76346047 refactor(rust): move OCR provider code into litellm-llms and delete core/src/llms 
OCR transformations, BaseOcrConfig with its response and connection types,
the OCR error, and the HTTP pieces (custom_httpx: http_handler, transport,
media, llm_http_handler with OcrClient and the request/response handler)
now live in litellm-llms at their Python paths. Provider code no longer
reaches into the route: it gets the caller's hooks through a route-neutral
CallHooks trait that core implements over its host, and core dispatches to
llm_http_handler::ocr with the concrete config, the way Python calls
base_llm_http_handler.ocr(provider_config=...).

Core keeps the route: entrypoint, request types, credential fallback,
provider dispatch, the machine and hook glue. ocr/mod.rs no longer
re-exports anything, provider constants moved next to their only users,
and provider tests that drive the whole route moved to core's route test
files. Twenty-two of those were exact copies of tests already there and
were dropped; every one still runs once.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-17 22:29:08 -07:00

2 KiB
Raw Blame History

litellm-core is the LiteLLM SDK in Rust — it makes the LLM call. Each top-level call is a module under src/<route>/ exposing a public entrypoint named after the route (messages::messages(), the Rust equivalent of litellm.messages()): you call it and get a typed non-streaming response back.

Crate layering

Each crate mirrors one top-level Python package, so a Rust path reads as its Python path with the crate name in place of the package directory. Dependencies only point down:

  • litellm-types mirrors litellm/types/: pure serde data, no I/O
  • litellm-core-utils mirrors litellm/litellm_core_utils/: pure helpers (provider resolution, prompt factory, call arguments), no network I/O
  • litellm-llms mirrors litellm/llms/: base_llm/<api>/transformation.rs, <provider>/<api>/transformation.rs, and custom_httpx/ (HTTP helpers and the OCR request handler)
  • litellm-core mirrors the route packages (litellm/ocr/, litellm/messages/, ...): entrypoints, route request types, provider dispatch, the route machine, and hooks

A route module owns the call entrypoint, route request types (*Request<'a>), credential fallback, provider dispatch, and the handler glue that runs a provider config. Provider code never imports from core; when it needs the caller's hooks mid-call it goes through litellm_llms::custom_httpx::llm_http_handler::CallHooks, which each route implements over its host. Import every item from its canonical path. Never re-export another crate's items or give an item a second public path; the only re-export allowed is a private submodule surfacing its item at its module root (mod error; pub use error::Error;). Handlers belong in core or llms, never in a host crate

Not here: serving HTTP (axum routes, extractors), config file reading, rollout state, databases, or callback execution of any kind. Core runs each route as a machine that yields host operations and call events; which integrations consume those events is the host's business.