scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-03 07:23:33 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 166
litellm/litellm-rust/crates/inference-responses/AGENTS.md
Nate Armstrong bcd80b6aa1
feat(rust): encode LiteLLM response ids on native responses (#46572) 
Native Responses now hands back the same managed ids as Python: the
resp_ id carries the provider and deployment, encrypted-content items get
encitem_ ids and wrapped content under affinity routing, and container ids
become cntr_ ids. Encoded encrypted-content items in the input are restored
before the provider request. The codec lives in inference-responses with
goldens taken from litellm/responses/utils.py
2026-10-18 20:52:34 -07:00

770 B
Raw Permalink Blame History

  • Shared base and layering rules: ../inference/AGENTS.md
  • This crate owns Responses API call orchestration. Payloads belong in litellm-llms-types, transformations in llms/src/base_llm/responses and llms/src/<provider>/responses
  • The HTTP route returns litellm_host::call::CallOutput: a completed response, or a stream head and chunks, cached through litellm_inference::caching::execute_streaming
  • WebSocket sessions (src/websocket.rs) stay separate from the HTTP call driver because a connection can accept multiple requests while receiving events
  • src/ids.rs owns the LiteLLM-managed ids handed back to callers (resp_, encitem_, litellm_enc:, cntr_) and must stay fully compatible with litellm/responses/utils.py