refactor(llm): introduce Codec trait seam + extract openai_compatible (#481)

## What

Introduces the `Codec` / `StreamDecoder` trait seam in `fabro-llm` and
extracts the OpenAI Chat Completions wire logic behind it as the first
conforming codec. Two commits:

1. **`codec/mod.rs`** — the pure translation contract (`encode` /
`decode_response` / `stream_decoder`, plus defaulted
`encode_count_tokens` / `decode_count_tokens` / `decode_error`) and its
data types (`CodecCtx`, `CodecParams`, `EncodedRequest`, `RawEvent`). A
codec knows *what the bytes say*; it owns no HTTP, auth, or base URL.
2. **`codec/openai_compatible/`** — the Chat Completions codec split
into `wire` / `translate` / `request` / `response` / `stream`.
`providers/openai_compatible.rs` shrinks from 1,608 → ~330 lines: a thin
transport shell that keeps the public
struct/builders/auth/`validate_request`, owns the streaming byte loop +
SSE `data:` framing, and delegates all translation to the codec. The two
hand-rolled stream unfolds collapse into one.

This is the first step of a gateway refactor that separates codec (wire
dialect) from transport/auth/route, so later work (Bedrock, OpenRouter)
becomes mostly config rather than parallel adapters.

## Behavior

No behavior change. The public adapter API
(`OpenAiCompatibleAdapter::new` / `with_name` / `with_catalog` / …) is
unchanged, and **all 126 wire snapshots pass without edits** — the
parity proof that the extracted codec produces byte-identical output.
The 29 in-module unit tests move into the codec submodules alongside the
code they exercise.

## On the trait

`openai_compatible` is the simplest dialect, so its `impl Codec` is just
three methods — count-tokens and error mapping inherit the defaults. The
contract is defined in full now (a scoped `dead_code` allow on
`codec/mod.rs` covers the seams the anthropic/openai/gemini codecs will
exercise in follow-up PRs) so those extractions only *override* methods,
never extend the trait.

Extracting a real codec refined two trait signatures vs. the initial
sketch: the canonical `Request` lives in `CodecCtx` (decoders need it
for tool-argument parsing and the stream model fallback), and the
header-parsed `rate_limit` threads into `decode_response` /
`stream_decoder`. `on_event` returns `Result` so dialect error events
propagate as stream errors.

## Tests

- `cargo nextest run -p fabro-llm` — 515 passed (incl. 126 wire
snapshots, unmodified)
- `cargo nextest run --workspace` — green
- fabro-agent `parity_matrix` (the frozen `OpenAiCompatibleAdapter`
contract) — green
- `cargo +nightly fmt --check` / `clippy --all-targets -- -D warnings` —
clean

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Scott Werner 2026-06-10 12:50:19 -04:00 committed by GitHub
parent bbce4c7a2f
commit 3985eaf1d7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 1688 additions and 1461 deletions

View file

@ -0,0 +1,169 @@
//! The codec seam: pure, sync translation between the canonical core
//! (`Request`/`Response`/`StreamEvent`) and a provider wire dialect.
//!
//! A codec knows *what the bytes say*. It does NOT know how they travel
//! (auth, base URL, retries, streaming transport) — that's the adapter/
//! transport layer. Everything a codec varies on arrives as data in
//! [`CodecCtx`] / [`CodecParams`]; codecs hold no per-request state.
//!
//! The trait is intentionally complete (count-tokens + error mapping have
//! defaults) so the per-dialect codecs that follow only ever *override*
//! methods, never extend the contract.
// The contract is defined in full now, but `openai_compatible` (the first
// codec) is the simplest dialect and does not exercise every seam: the
// `model`/`params` context, `RawEvent.event`, and the count-tokens methods
// are consumed by the anthropic/openai/gemini codecs and the transport
// consolidation in later PRs of this series. Scoped to this trait-definition
// file; the codec impls below are fully used.
#![allow(
dead_code,
reason = "Codec contract is defined in full ahead of the dialects (PRs 3-6) that exercise \
the capability context, SSE event type, and count-tokens routes."
)]
pub(crate) mod openai_compatible;
use fabro_model::Model;
use crate::error::{Error, error_from_status_code};
use crate::providers::common::parse_error_body;
use crate::types::{RateLimitInfo, Request, Response, StreamEvent};
/// Per-request context. Borrowed — the codec reads what it needs and returns.
pub(crate) struct CodecCtx<'a> {
/// The canonical request being translated. Decoders read it too
/// (e.g. tool-argument parsing keys off the request's tool definitions;
/// the stream model fallback uses `request.model`).
pub request: &'a Request,
/// Identity stamped into `Response.provider`, and the `provider_options`
/// namespace key for the openai_compatible codec (kimi/zai/…).
pub provider_name: &'a str,
/// The model id to send on the wire — catalog `api_id`, resolved by the
/// route (today `api_id == id` everywhere).
pub deployment_id: &'a str,
/// Model row for capability lookups (prompt_cache, reasoning levels,
/// max_output). `None` when no catalog is injected.
pub model: Option<&'a Model>,
/// Per-route dialect data (model/version placement, …). Defaulted to
/// today's direct-route values; Bedrock/OpenRouter add variants later.
pub params: &'a CodecParams,
}
/// Per-route dialect knobs, expressed as data so one codec can serve several
/// routes. Starts empty; grows by adding `#[serde(default)]` fields (a
/// non-breaking change) — e.g. PR 3 adds version-placement, #459 adds
/// model-placement for Bedrock.
#[derive(Debug, Default, Clone)]
pub(crate) struct CodecParams;
/// What [`Codec::encode`] produces. The transport applies `endpoint` +
/// `headers` on top of the route's base URL and auth; the codec never touches
/// HTTP.
pub(crate) struct EncodedRequest {
/// Request body.
pub body: serde_json::Value,
/// Path appended to the route base URL, fully formed by the codec
/// (incl. model-in-path and `?alt=sse` for gemini). e.g.
/// `/chat/completions`.
pub endpoint: String,
/// Dialect headers as data (e.g. `anthropic-version`, beta headers).
/// NOT auth or `content-type` — those are the transport's job. Empty for
/// the openai_compatible codec.
pub headers: Vec<(String, String)>,
}
/// One framed item off the byte stream, handed to a [`StreamDecoder`].
pub(crate) struct RawEvent<'a> {
/// SSE `event:` type — `Some` for anthropic; `None` for the data-only
/// framing openai/gemini use.
pub event: Option<&'a str>,
/// The `data:` payload, or a bare JSON line. The sentinel `[DONE]` is
/// passed through verbatim for the decoder to recognize.
pub data: &'a str,
}
/// Stateless translator for one wire dialect.
pub(crate) trait Codec: Send + Sync {
/// Canonical request (`ctx.request`) → wire request. `stream` selects the
/// streaming shape (`stream: true` in the body, gemini's
/// `:streamGenerateContent` endpoint). Fallible: attachment/parameter
/// encoding can reject.
fn encode(&self, ctx: &CodecCtx<'_>, stream: bool) -> Result<EncodedRequest, Error>;
/// Wire response body → canonical `Response` (content parts, finish
/// reason, usage). Each dialect's finish-reason map and usage arithmetic
/// live here. Stamps `ctx.provider_name` into `Response.provider` and the
/// transport-parsed `rate_limit` into the response.
fn decode_response(
&self,
body: &str,
ctx: &CodecCtx<'_>,
rate_limit: Option<RateLimitInfo>,
) -> Result<Response, Error>;
/// A fresh stateful decoder for one streaming response. `rate_limit` is the
/// transport-parsed header value to embed in the synthesized `Finish`.
fn stream_decoder(
&self,
ctx: &CodecCtx<'_>,
rate_limit: Option<RateLimitInfo>,
) -> Box<dyn StreamDecoder>;
/// The third route, if the dialect has one (`/messages/count_tokens`,
/// `/responses/input_tokens`, `:countTokens`). `None` = the dialect has no
/// such route. Whether a given *deployment* may use it is a separate
/// route-level gate (Kimi-over-anthropic) decided before this is called.
fn encode_count_tokens(&self, _ctx: &CodecCtx<'_>) -> Option<Result<EncodedRequest, Error>> {
None
}
/// Parse the token count out of a count-tokens response. Only called when
/// [`Codec::encode_count_tokens`] returned `Some`; the default guards the
/// invariant for codecs without a count route.
fn decode_count_tokens(&self, _body: &str) -> Result<i64, Error> {
Err(Error::Configuration {
message: "codec has no count_tokens route".to_string(),
source: None,
})
}
/// Map a non-2xx response to an `Error`. `retry_after` is the
/// transport-parsed `retry-after` header value in seconds (header parsing
/// is the transport's job, like `rate_limit` on the decode methods).
/// Default = shared HTTP-status mapping (what openai_compatible uses);
/// anthropic/openai/gemini override to fold in dialect error bodies
/// (error.type, gRPC status, …).
fn decode_error(
&self,
status: u16,
body: &str,
ctx: &CodecCtx<'_>,
retry_after: Option<f64>,
) -> Error {
let (message, code, raw) = parse_error_body(body, "type");
error_from_status_code(
status,
message,
ctx.provider_name.to_string(),
code,
raw,
retry_after,
)
}
}
/// Stateful per-stream decoder, driven by the shared transport loop.
/// `'static` because it is boxed into the stream's unfold state.
pub(crate) trait StreamDecoder: Send + 'static {
/// One framed event → zero or more canonical `StreamEvent`s. Returns
/// `Err` for dialect error events (anthropic `error`, openai
/// `response.failed`), which the transport yields as a stream error.
fn on_event(&mut self, ev: RawEvent<'_>) -> Result<Vec<StreamEvent>, Error>;
/// Byte-stream-end hook. Semantics are per-decoder, not shared:
/// anthropic — return nothing (`message_stop` already finished it);
/// openai_compatible — synthesize `Finish` iff content started (minimax);
/// gemini — synthesize `Finish` unconditionally if not yet finished.
fn finish(&mut self) -> Vec<StreamEvent>;
}

View file

@ -0,0 +1,43 @@
//! The OpenAI Chat Completions (`/chat/completions`) codec.
//!
//! Serves every "OpenAI-compatible" route (kimi, zai, minimax, venice,
//! inception, ollama, litellm, …). Pure translation: no HTTP, auth, or base
//! URL — the adapter shell owns those. Count-tokens and error mapping use the
//! `Codec` trait defaults (this dialect has no count route and uses the shared
//! HTTP-status error mapping).
mod request;
mod response;
mod stream;
mod translate;
mod wire;
use crate::codec::{Codec, CodecCtx, EncodedRequest, StreamDecoder};
use crate::error::Error;
use crate::types::{RateLimitInfo, Response};
/// Codec for the OpenAI Chat Completions wire dialect.
pub(crate) struct OpenAiCompatible;
impl Codec for OpenAiCompatible {
fn encode(&self, ctx: &CodecCtx<'_>, stream: bool) -> Result<EncodedRequest, Error> {
Ok(request::encode(ctx, stream))
}
fn decode_response(
&self,
body: &str,
ctx: &CodecCtx<'_>,
rate_limit: Option<RateLimitInfo>,
) -> Result<Response, Error> {
response::decode_response(body, ctx, rate_limit)
}
fn stream_decoder(
&self,
ctx: &CodecCtx<'_>,
rate_limit: Option<RateLimitInfo>,
) -> Box<dyn StreamDecoder> {
Box::new(stream::StreamState::new(ctx, rate_limit))
}
}

View file

@ -0,0 +1,253 @@
//! Request encoding: canonical `Request` → Chat Completions body.
use super::translate;
use super::wire::ApiRequest;
use crate::codec::{CodecCtx, EncodedRequest};
/// Build the Chat Completions request for `ctx.request`. `stream` toggles the
/// `stream` body field. The body is assembled as a `serde_json::Value` so
/// `provider_options.<provider_name>` fields can be merged in before sending.
///
/// Infallible for this dialect — the `Codec::encode` `Result` is wrapped by the
/// trait impl.
pub(super) fn encode(ctx: &CodecCtx<'_>, stream: bool) -> EncodedRequest {
let request = ctx.request;
let chat_messages = translate::translate_messages(&request.messages);
let tools = request
.tools
.as_ref()
.map(|t| translate::translate_tools(t));
let tool_choice = request
.tool_choice
.as_ref()
.map(translate::translate_tool_choice);
let response_format = request
.response_format
.as_ref()
.map(translate::translate_response_format);
let api_request = ApiRequest {
model: ctx.deployment_id.to_string(),
messages: chat_messages,
temperature: request.temperature,
max_tokens: request.max_tokens,
top_p: request.top_p,
stop: request.stop_sequences.clone(),
tools,
tool_choice,
response_format,
stream: stream.then_some(true),
};
let mut body = serde_json::to_value(&api_request).unwrap_or_default();
merge_provider_options(
&mut body,
request.provider_options.as_ref(),
ctx.provider_name,
);
EncodedRequest {
body,
endpoint: "/chat/completions".to_string(),
headers: Vec::new(),
}
}
/// Merge `provider_options.<provider_name>` fields into the serialized API
/// request body.
///
/// The provider name is configurable (e.g. "groq", "together", "kimi"),
/// allowing each instance to have its own namespace in `provider_options`.
pub(super) fn merge_provider_options(
body: &mut serde_json::Value,
provider_options: Option<&serde_json::Value>,
provider_name: &str,
) {
let Some(opts) = provider_options.and_then(|opts| opts.get(provider_name)) else {
return;
};
let Some(body_map) = body.as_object_mut() else {
return;
};
let Some(opts_map) = opts.as_object() else {
return;
};
for (key, value) in opts_map {
body_map.insert(key.clone(), value.clone());
}
}
#[cfg(test)]
mod tests {
use super::super::wire::ApiRequest;
use super::*;
use crate::codec::CodecParams;
use crate::types::{Message, Request};
fn minimal_request() -> Request {
Request {
model: "llama-3.1-70b".to_string(),
messages: vec![Message::user("Hello")],
provider: None,
tools: None,
tool_choice: None,
response_format: None,
temperature: None,
top_p: None,
max_tokens: None,
stop_sequences: None,
reasoning_effort: None,
speed: None,
metadata: None,
provider_options: None,
}
}
/// Encode `request` through the codec with `deployment_id == request.model`
/// (the no-catalog case) and return the body.
fn encode_body(request: &Request, provider_name: &str, stream: bool) -> serde_json::Value {
let params = CodecParams;
let deployment_id = request.model.clone();
let ctx = CodecCtx {
request,
provider_name,
deployment_id: &deployment_id,
model: None,
params: &params,
};
encode(&ctx, stream).body
}
#[test]
fn api_request_stream_field_serialization() {
let req = ApiRequest {
model: "test".into(),
messages: vec![],
temperature: None,
max_tokens: None,
top_p: None,
stop: None,
tools: None,
tool_choice: None,
response_format: None,
stream: Some(true),
};
let json = serde_json::to_value(&req).unwrap();
assert_eq!(json["stream"], true);
let req_no_stream = ApiRequest {
model: "test".into(),
messages: vec![],
temperature: None,
max_tokens: None,
top_p: None,
stop: None,
tools: None,
tool_choice: None,
response_format: None,
stream: None,
};
let json_no_stream = serde_json::to_value(&req_no_stream).unwrap();
assert!(json_no_stream.get("stream").is_none());
}
#[test]
fn encode_uses_deployment_id_as_model() {
let request = minimal_request();
let params = CodecParams;
let deployment_id = "acme/model-large".to_string();
let ctx = CodecCtx {
request: &request,
provider_name: "acme",
deployment_id: &deployment_id,
model: None,
params: &params,
};
let body = encode(&ctx, false).body;
assert_eq!(body["model"], "acme/model-large");
}
#[test]
fn provider_options_none_produces_standard_body() {
let request = minimal_request();
let body = encode_body(&request, "groq", false);
assert_eq!(body["model"], "llama-3.1-70b");
assert!(body.get("stream").is_none());
}
#[test]
fn provider_options_matching_name_merged() {
let mut request = minimal_request();
request.provider_options = Some(serde_json::json!({
"groq": {
"frequency_penalty": 0.5,
"presence_penalty": 0.3
}
}));
let body = encode_body(&request, "groq", false);
assert_eq!(body["frequency_penalty"], 0.5);
assert_eq!(body["presence_penalty"], 0.3);
}
#[test]
fn provider_options_different_name_ignored() {
let mut request = minimal_request();
request.provider_options = Some(serde_json::json!({
"together": {
"repetition_penalty": 1.2
}
}));
let body = encode_body(&request, "groq", false);
assert!(body.get("repetition_penalty").is_none());
}
#[test]
fn provider_options_uses_adapter_name() {
let mut request = minimal_request();
request.provider_options = Some(serde_json::json!({
"together": {
"repetition_penalty": 1.2
}
}));
let body = encode_body(&request, "together", false);
assert_eq!(body["repetition_penalty"], 1.2);
}
#[test]
fn provider_options_preserves_standard_fields() {
let mut request = minimal_request();
request.temperature = Some(0.7);
request.max_tokens = Some(200);
request.provider_options = Some(serde_json::json!({
"groq": {
"frequency_penalty": 0.5
}
}));
let body = encode_body(&request, "groq", true);
assert_eq!(body["temperature"], 0.7);
assert_eq!(body["max_tokens"], 200);
assert_eq!(body["stream"], true);
assert_eq!(body["frequency_penalty"], 0.5);
}
#[test]
fn provider_options_can_override_model() {
let mut request = minimal_request();
request.provider_options = Some(serde_json::json!({
"groq": {
"model": "custom-model"
}
}));
let body = encode_body(&request, "groq", false);
assert_eq!(body["model"], "custom-model");
}
#[test]
fn merge_provider_options_with_non_object_value() {
let mut body = serde_json::json!({"model": "test"});
let opts = serde_json::json!({"groq": "not-an-object"});
merge_provider_options(&mut body, Some(&opts), "groq");
assert_eq!(body["model"], "test");
}
}

View file

@ -0,0 +1,83 @@
//! Response decoding: Chat Completions body → canonical `Response`.
use super::translate::{self, map_finish_reason};
use super::wire::ApiResponse;
use crate::codec::CodecCtx;
use crate::error::{Error, ProviderErrorDetail, ProviderErrorKind};
use crate::types::{
ContentPart, Message, RateLimitInfo, Response, Role, ThinkingData, TokenCounts, ToolCall,
};
pub(super) fn decode_response(
body: &str,
ctx: &CodecCtx<'_>,
rate_limit: Option<RateLimitInfo>,
) -> Result<Response, Error> {
let api_resp: ApiResponse = serde_json::from_str(body)
.map_err(|e| Error::network(format!("failed to parse response: {e}"), e))?;
let choice = api_resp.choices.first().ok_or_else(|| Error::Provider {
kind: ProviderErrorKind::Server,
detail: Box::new(ProviderErrorDetail::new(
"no choices in response",
ctx.provider_name,
)),
})?;
let mut content_parts = Vec::new();
if let Some(reasoning) = &choice.message.reasoning_content {
if !reasoning.is_empty() {
content_parts.push(ContentPart::Thinking(ThinkingData {
text: reasoning.clone(),
signature: None,
redacted: false,
}));
}
}
if let Some(text) = &choice.message.content {
if !text.is_empty() {
content_parts.push(ContentPart::text(text));
}
}
if let Some(tool_calls) = &choice.message.tool_calls {
let custom_tool_names = translate::custom_tool_names(ctx.request);
for tc in tool_calls {
let arguments = translate::parse_tool_arguments(
&tc.function.name,
&tc.function.arguments,
&custom_tool_names,
);
let mut tool_call = ToolCall::new(&tc.id, &tc.function.name, arguments);
tool_call.raw_arguments = Some(tc.function.arguments.clone());
content_parts.push(ContentPart::ToolCall(tool_call));
}
}
let finish_reason = map_finish_reason(choice.finish_reason.as_deref());
let usage = api_resp
.usage
.as_ref()
.map_or_else(TokenCounts::default, |u| TokenCounts {
input_tokens: u.prompt_tokens,
output_tokens: u.completion_tokens,
..TokenCounts::default()
});
Ok(Response {
id: api_resp.id,
model: api_resp.model,
provider: ctx.provider_name.to_string(),
message: Message {
role: Role::Assistant,
content: content_parts,
name: None,
tool_call_id: None,
},
finish_reason,
usage,
raw: serde_json::from_str(body).ok(),
warnings: vec![],
rate_limit,
})
}

View file

@ -0,0 +1,475 @@
//! Streaming decoder: Chat Completions SSE chunks → canonical `StreamEvent`s.
//!
//! Byte reading and `data:` framing live in the transport; this decoder is fed
//! already-stripped payloads (including the `[DONE]` sentinel) via `on_event`.
use super::translate::{map_finish_reason, parse_tool_arguments};
use super::wire::{AccumulatedToolCall, StreamChunk};
use crate::codec::{CodecCtx, RawEvent, StreamDecoder};
use crate::error::Error;
use crate::types::{
ContentPart, FinishReason, Message, RateLimitInfo, Response, Role, StreamEvent, ThinkingData,
TokenCounts, ToolCall,
};
/// Accumulated state while decoding the Chat Completions SSE stream.
pub(super) struct StreamState {
provider_name: String,
model: String,
response_id: String,
response_model: String,
accumulated_text: String,
accumulated_reasoning: String,
tool_calls: Vec<AccumulatedToolCall>,
usage: TokenCounts,
finish_reason: FinishReason,
text_started: bool,
custom_tool_names: Vec<String>,
/// True after `finish_events()` has run (guards against duplicates).
finished: bool,
rate_limit: Option<RateLimitInfo>,
}
impl StreamState {
pub(super) fn new(ctx: &CodecCtx<'_>, rate_limit: Option<RateLimitInfo>) -> Self {
Self {
provider_name: ctx.provider_name.to_string(),
model: ctx.request.model.clone(),
response_id: String::new(),
response_model: String::new(),
accumulated_text: String::new(),
accumulated_reasoning: String::new(),
tool_calls: Vec::new(),
usage: TokenCounts::default(),
finish_reason: FinishReason::Stop,
text_started: false,
custom_tool_names: super::translate::custom_tool_names(ctx.request),
finished: false,
rate_limit,
}
}
/// Process a parsed SSE chunk and return events to emit, if any.
fn process_chunk(&mut self, chunk: &StreamChunk) -> Option<Vec<StreamEvent>> {
// Capture response metadata from the first chunk.
if let Some(id) = &chunk.id {
if self.response_id.is_empty() {
self.response_id.clone_from(id);
}
}
if let Some(model) = &chunk.model {
if self.response_model.is_empty() {
self.response_model.clone_from(model);
}
}
// Capture usage if present (often in a dedicated chunk).
if let Some(usage) = &chunk.usage {
self.usage = TokenCounts {
input_tokens: usage.prompt_tokens,
output_tokens: usage.completion_tokens,
..TokenCounts::default()
};
}
let choices = chunk.choices.as_ref()?;
let choice = choices.first()?;
let mut events = Vec::new();
// Check for finish_reason.
if let Some(reason) = &choice.finish_reason {
self.finish_reason = map_finish_reason(Some(reason.as_str()));
}
let delta = choice.delta.as_ref()?;
// Accumulate reasoning/thinking content (Kimi, etc.).
if let Some(reasoning) = &delta.reasoning_content {
if !reasoning.is_empty() {
self.accumulated_reasoning.push_str(reasoning);
}
}
// Handle text content delta.
if let Some(content) = &delta.content {
if !content.is_empty() {
if !self.text_started {
self.text_started = true;
events.push(StreamEvent::TextStart { text_id: None });
}
self.accumulated_text.push_str(content);
events.push(StreamEvent::text_delta(content, None));
}
}
// Handle tool call deltas.
if let Some(tool_calls) = &delta.tool_calls {
for tc in tool_calls {
let index = tc.index;
// Grow the accumulated tool calls vector if needed.
while self.tool_calls.len() <= index {
self.tool_calls.push(AccumulatedToolCall {
id: String::new(),
name: String::new(),
arguments: String::new(),
started: false,
});
}
let accumulated = &mut self.tool_calls[index];
// First chunk for this tool call carries id and name.
if let Some(id) = &tc.id {
accumulated.id.clone_from(id);
}
if let Some(func) = &tc.function {
if let Some(name) = &func.name {
accumulated.name.clone_from(name);
}
if let Some(args) = &func.arguments {
accumulated.arguments.push_str(args);
}
}
let partial_tool_call =
ToolCall::new(&accumulated.id, &accumulated.name, serde_json::json!(null));
if accumulated.started {
events.push(StreamEvent::ToolCallDelta {
tool_call: partial_tool_call,
});
} else {
accumulated.started = true;
events.push(StreamEvent::ToolCallStart {
tool_call: partial_tool_call,
});
}
}
}
if events.is_empty() {
None
} else {
Some(events)
}
}
/// Generate the final events when `[DONE]` (or end-of-stream) is received.
fn finish_events(&mut self) -> Vec<StreamEvent> {
self.finished = true;
let mut events = Vec::new();
// End text segment if it was started.
if self.text_started {
events.push(StreamEvent::TextEnd { text_id: None });
}
let mut content_parts = Vec::new();
// Include reasoning/thinking content if present (Kimi, etc.).
if !self.accumulated_reasoning.is_empty() {
content_parts.push(ContentPart::Thinking(ThinkingData {
text: std::mem::take(&mut self.accumulated_reasoning),
signature: None,
redacted: false,
}));
}
if !self.accumulated_text.is_empty() {
content_parts.push(ContentPart::text(&self.accumulated_text));
}
for accumulated in &self.tool_calls {
let arguments = parse_tool_arguments(
&accumulated.name,
&accumulated.arguments,
&self.custom_tool_names,
);
let mut tool_call = ToolCall::new(&accumulated.id, &accumulated.name, arguments);
tool_call.raw_arguments = Some(accumulated.arguments.clone());
events.push(StreamEvent::ToolCallEnd {
tool_call: tool_call.clone(),
});
content_parts.push(ContentPart::ToolCall(tool_call));
}
// Infer finish reason from tool calls if not explicitly set.
if !self.tool_calls.is_empty() && self.finish_reason == FinishReason::Stop {
self.finish_reason = FinishReason::ToolCalls;
}
let response_model = if self.response_model.is_empty() {
self.model.clone()
} else {
self.response_model.clone()
};
let response = Response {
id: self.response_id.clone(),
model: response_model,
provider: self.provider_name.clone(),
message: Message {
role: Role::Assistant,
content: content_parts,
name: None,
tool_call_id: None,
},
finish_reason: self.finish_reason.clone(),
usage: self.usage.clone(),
raw: None,
warnings: vec![],
rate_limit: self.rate_limit.clone(),
};
events.push(StreamEvent::finish(
self.finish_reason.clone(),
self.usage.clone(),
response,
));
events
}
}
impl StreamDecoder for StreamState {
fn on_event(&mut self, ev: RawEvent<'_>) -> Result<Vec<StreamEvent>, Error> {
// Chat Completions uses data-only framing; the `event:` field is unused.
if ev.data == "[DONE]" {
return Ok(self.finish_events());
}
let chunk: StreamChunk = serde_json::from_str(ev.data)
.map_err(|e| Error::stream_error(format!("failed to parse SSE chunk: {e}"), e))?;
Ok(self.process_chunk(&chunk).unwrap_or_default())
}
fn finish(&mut self) -> Vec<StreamEvent> {
// Stream ended without `[DONE]`. Some providers (e.g. Minimax) omit the
// sentinel; emit accumulated finish events if we have content and
// haven't already finished.
if !self.finished && (self.text_started || !self.tool_calls.is_empty()) {
self.finish_events()
} else {
Vec::new()
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::codec::CodecParams;
use crate::types::Request;
/// Build a decoder through `StreamState::new` (with a minimal request) for
/// unit tests that drive `process_chunk` / `finish_events`.
fn test_state(provider: &str, model: &str) -> StreamState {
let request = Request {
model: model.to_string(),
messages: Vec::new(),
provider: None,
tools: None,
tool_choice: None,
response_format: None,
temperature: None,
top_p: None,
max_tokens: None,
stop_sequences: None,
reasoning_effort: None,
speed: None,
metadata: None,
provider_options: None,
};
let params = CodecParams;
let ctx = CodecCtx {
request: &request,
provider_name: provider,
deployment_id: model,
model: None,
params: &params,
};
StreamState::new(&ctx, None)
}
#[test]
fn stream_chunk_minimax_format() {
let json = r#"{"id":"abc","choices":[{"index":0,"delta":{"content":"hello","role":"assistant","name":"MiniMax AI","audio_content":""}}],"created":1772268546,"model":"MiniMax-M2.5","object":"chat.completion.chunk","usage":null,"input_sensitive":false,"output_sensitive":false}"#;
let chunk: StreamChunk = serde_json::from_str(json).unwrap();
let choices = chunk.choices.unwrap();
let delta = choices[0].delta.as_ref().unwrap();
assert_eq!(delta.content.as_deref(), Some("hello"));
}
#[test]
fn stream_chunk_text_delta_parsing() {
let json = r#"{"id":"chatcmpl-1","model":"gpt-4","choices":[{"delta":{"content":"Hello"},"finish_reason":null}]}"#;
let chunk: StreamChunk = serde_json::from_str(json).unwrap();
assert_eq!(chunk.id.as_deref(), Some("chatcmpl-1"));
assert_eq!(chunk.model.as_deref(), Some("gpt-4"));
let choices = chunk.choices.unwrap();
assert_eq!(choices.len(), 1);
let delta = choices[0].delta.as_ref().unwrap();
assert_eq!(delta.content.as_deref(), Some("Hello"));
assert!(choices[0].finish_reason.is_none());
}
#[test]
fn stream_chunk_tool_call_parsing() {
let json = r#"{"id":"chatcmpl-1","model":"gpt-4","choices":[{"delta":{"tool_calls":[{"index":0,"id":"call_1","function":{"name":"get_weather","arguments":"{\"ci"}}]},"finish_reason":null}]}"#;
let chunk: StreamChunk = serde_json::from_str(json).unwrap();
let choices = chunk.choices.unwrap();
let delta = choices[0].delta.as_ref().unwrap();
let tc = &delta.tool_calls.as_ref().unwrap()[0];
assert_eq!(tc.index, 0);
assert_eq!(tc.id.as_deref(), Some("call_1"));
let func = tc.function.as_ref().unwrap();
assert_eq!(func.name.as_deref(), Some("get_weather"));
assert_eq!(func.arguments.as_deref(), Some("{\"ci"));
}
#[test]
fn stream_chunk_usage_parsing() {
let json = r#"{"id":"chatcmpl-1","model":"gpt-4","choices":[],"usage":{"prompt_tokens":10,"completion_tokens":20,"total_tokens":30}}"#;
let chunk: StreamChunk = serde_json::from_str(json).unwrap();
let usage = chunk.usage.unwrap();
assert_eq!(usage.prompt_tokens, 10);
assert_eq!(usage.completion_tokens, 20);
}
#[test]
fn stream_chunk_finish_reason_parsing() {
let json = r#"{"id":"chatcmpl-1","model":"gpt-4","choices":[{"delta":{},"finish_reason":"stop"}]}"#;
let chunk: StreamChunk = serde_json::from_str(json).unwrap();
let choices = chunk.choices.unwrap();
assert_eq!(choices[0].finish_reason.as_deref(), Some("stop"));
}
#[test]
fn process_text_chunks() {
let mut state = test_state("test", "model");
let chunk1: StreamChunk = serde_json::from_str(
r#"{"id":"c1","model":"m1","choices":[{"delta":{"content":"Hello"},"finish_reason":null}]}"#,
).unwrap();
let events1 = state.process_chunk(&chunk1).unwrap();
assert_eq!(events1.len(), 2);
assert!(matches!(events1[0], StreamEvent::TextStart { .. }));
assert!(matches!(events1[1], StreamEvent::TextDelta { .. }));
let chunk2: StreamChunk = serde_json::from_str(
r#"{"id":"c1","model":"m1","choices":[{"delta":{"content":" world"},"finish_reason":null}]}"#,
).unwrap();
let events2 = state.process_chunk(&chunk2).unwrap();
assert_eq!(events2.len(), 1);
assert!(matches!(events2[0], StreamEvent::TextDelta { .. }));
assert_eq!(state.accumulated_text, "Hello world");
}
#[test]
fn process_tool_call_chunks() {
let mut state = test_state("test", "model");
let chunk1: StreamChunk = serde_json::from_str(
r#"{"id":"c1","model":"m1","choices":[{"delta":{"tool_calls":[{"index":0,"id":"call_1","function":{"name":"fn1","arguments":"{\"k"}}]},"finish_reason":null}]}"#,
).unwrap();
let events1 = state.process_chunk(&chunk1).unwrap();
assert_eq!(events1.len(), 1);
assert!(matches!(events1[0], StreamEvent::ToolCallStart { .. }));
let chunk2: StreamChunk = serde_json::from_str(
r#"{"id":"c1","model":"m1","choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"ey\"}"}}]},"finish_reason":null}]}"#,
).unwrap();
let events2 = state.process_chunk(&chunk2).unwrap();
assert_eq!(events2.len(), 1);
assert!(matches!(events2[0], StreamEvent::ToolCallDelta { .. }));
assert_eq!(state.tool_calls[0].arguments, r#"{"key"}"#);
}
#[test]
fn finish_events_text_only() {
let mut state = test_state("test-provider", "test-model");
state.response_id = "resp-1".into();
state.response_model = "gpt-4".into();
state.accumulated_text = "Hello world".into();
state.text_started = true;
state.usage = TokenCounts {
input_tokens: 5,
output_tokens: 10,
..TokenCounts::default()
};
let events = state.finish_events();
assert_eq!(events.len(), 2);
assert!(matches!(events[0], StreamEvent::TextEnd { .. }));
match &events[1] {
StreamEvent::Finish {
finish_reason,
usage,
response,
} => {
assert_eq!(*finish_reason, FinishReason::Stop);
assert_eq!(usage.input_tokens, 5);
assert_eq!(usage.output_tokens, 10);
assert_eq!(response.text(), "Hello world");
assert_eq!(response.id, "resp-1");
assert_eq!(response.model, "gpt-4");
assert_eq!(response.provider, "test-provider");
}
other => panic!("Expected Finish, got {other:?}"),
}
}
#[test]
fn finish_events_with_tool_calls() {
let mut state = test_state("test", "model");
state.response_id = "resp-1".into();
state.tool_calls.push(AccumulatedToolCall {
id: "call_1".into(),
name: "get_weather".into(),
arguments: r#"{"city":"SF"}"#.into(),
started: true,
});
let events = state.finish_events();
assert_eq!(events.len(), 2);
match &events[0] {
StreamEvent::ToolCallEnd { tool_call } => {
assert_eq!(tool_call.id, "call_1");
assert_eq!(tool_call.name, "get_weather");
assert_eq!(tool_call.raw_arguments.as_deref(), Some(r#"{"city":"SF"}"#));
}
other => panic!("Expected ToolCallEnd, got {other:?}"),
}
match &events[1] {
StreamEvent::Finish {
finish_reason,
response,
..
} => {
assert_eq!(*finish_reason, FinishReason::ToolCalls);
let calls = response.tool_calls();
assert_eq!(calls.len(), 1);
assert_eq!(calls[0].name, "get_weather");
}
other => panic!("Expected Finish, got {other:?}"),
}
}
#[test]
fn uses_request_model_as_fallback() {
let mut state = test_state("test", "fallback-model");
let events = state.finish_events();
match &events[0] {
StreamEvent::Finish { response, .. } => {
assert_eq!(response.model, "fallback-model");
}
other => panic!("Expected Finish, got {other:?}"),
}
}
}

View file

@ -0,0 +1,411 @@
//! Pure mapping between canonical types and the Chat Completions wire shapes.
use super::wire::{ChatFunction, ChatMessage, ChatToolCall};
use crate::types::{
ContentPart, FinishReason, Message, Request, ResponseFormat, ResponseFormatType, Role,
ToolChoice, ToolDefinition,
};
pub(super) fn map_finish_reason(reason: Option<&str>) -> FinishReason {
match reason {
Some("stop") | None => FinishReason::Stop,
Some("length") => FinishReason::Length,
Some("tool_calls") => FinishReason::ToolCalls,
Some("content_filter") => FinishReason::ContentFilter,
Some(other) => FinishReason::Other(other.to_string()),
}
}
/// Build the content string from a message's parts, including fallback text
/// for unsupported content types (Audio, Document).
fn content_text_with_fallbacks(parts: &[ContentPart]) -> String {
let mut segments: Vec<String> = Vec::new();
for part in parts {
match part {
ContentPart::Text(text) => segments.push(text.clone()),
ContentPart::Audio(_) => {
segments.push("[Audio content not supported by this provider]".to_string());
}
ContentPart::Document(doc) => {
let desc = doc.file_name.as_ref().map_or_else(
|| "[Document content not supported by this provider]".to_string(),
|name| {
format!("[Document '{name}': content type not supported by this provider]")
},
);
segments.push(desc);
}
_ => {}
}
}
segments.join("")
}
pub(super) fn translate_messages(messages: &[Message]) -> Vec<ChatMessage> {
messages
.iter()
.flat_map(|msg| {
// Tool messages must be split into one ChatMessage per ToolResult,
// each with its own tool_call_id. The Chat Completions API requires
// every tool_call_id from the assistant to have a matching tool message.
if msg.role == Role::Tool {
return msg
.content
.iter()
.filter_map(|part| {
if let ContentPart::ToolResult(tr) = part {
let output = tr
.content
.as_str()
.map_or_else(|| tr.content.to_string(), str::to_string);
Some(ChatMessage {
role: "tool".to_string(),
content: Some(output),
reasoning_content: None,
tool_call_id: Some(tr.tool_call_id.clone()),
tool_calls: None,
})
} else {
None
}
})
.collect::<Vec<_>>();
}
let role = match msg.role {
Role::System | Role::Developer => "system",
Role::User => "user",
Role::Assistant => "assistant",
Role::Tool => unreachable!(
"Role::Tool is handled in the early-return branch above this match"
),
};
let mut tool_calls: Vec<ChatToolCall> = Vec::new();
if msg.role == Role::Assistant {
for part in &msg.content {
if let ContentPart::ToolCall(tc) = part {
let arguments = tc
.raw_arguments
.clone()
.unwrap_or_else(|| tc.arguments.to_string());
tool_calls.push(ChatToolCall {
id: tc.id.clone(),
kind: "function".to_string(),
function: ChatFunction {
name: tc.name.clone(),
arguments,
},
});
}
}
}
let text = content_text_with_fallbacks(&msg.content);
let content = if text.is_empty() { None } else { Some(text) };
let tool_calls = if tool_calls.is_empty() {
None
} else {
Some(tool_calls)
};
// Extract reasoning/thinking content for assistant messages.
let reasoning_content = if msg.role == Role::Assistant {
let reasoning: String = msg
.content
.iter()
.filter_map(|part| match part {
ContentPart::Thinking(t) if !t.redacted => Some(t.text.as_str()),
_ => None,
})
.collect::<Vec<_>>()
.join("");
if reasoning.is_empty() {
None
} else {
Some(reasoning)
}
} else {
None
};
vec![ChatMessage {
role: role.to_string(),
content,
reasoning_content,
tool_call_id: msg.tool_call_id.clone(),
tool_calls,
}]
})
.collect()
}
pub(super) fn translate_tools(tools: &[ToolDefinition]) -> Vec<serde_json::Value> {
tools
.iter()
.map(|t| {
serde_json::json!({
"type": "function",
"function": {
"name": t.name,
"description": t.description,
"parameters": t.parameters,
}
})
})
.collect()
}
pub(super) fn translate_tool_choice(choice: &ToolChoice) -> serde_json::Value {
match choice {
ToolChoice::Auto => serde_json::json!("auto"),
ToolChoice::None => serde_json::json!("none"),
ToolChoice::Required => serde_json::json!("required"),
ToolChoice::Named { tool_name } => {
serde_json::json!({"type": "function", "function": {"name": tool_name}})
}
}
}
pub(super) fn custom_tool_names(request: &Request) -> Vec<String> {
request
.tools
.as_deref()
.unwrap_or_default()
.iter()
.filter(|tool| tool.is_custom())
.map(|tool| tool.name.clone())
.collect()
}
pub(super) fn parse_tool_arguments(
tool_name: &str,
raw_arguments: &str,
custom_tool_names: &[String],
) -> serde_json::Value {
match serde_json::from_str(raw_arguments) {
Ok(arguments) => arguments,
Err(_) if custom_tool_names.iter().any(|name| name == tool_name) => {
serde_json::Value::String(raw_arguments.to_string())
}
Err(_) => serde_json::json!({}),
}
}
/// Translate unified `ResponseFormat` to Chat Completions `response_format`.
pub(super) fn translate_response_format(format: &ResponseFormat) -> serde_json::Value {
match format.kind {
ResponseFormatType::Text => serde_json::json!({"type": "text"}),
ResponseFormatType::JsonObject => serde_json::json!({"type": "json_object"}),
ResponseFormatType::JsonSchema => {
let mut json_schema = serde_json::json!({
"name": "response",
"strict": format.strict,
});
if let Some(schema) = &format.json_schema {
json_schema["schema"] = schema.clone();
}
serde_json::json!({
"type": "json_schema",
"json_schema": json_schema,
})
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::{AudioData, ContentPart, DocumentData, Message, Role, ToolCall};
#[test]
fn translate_assistant_message_with_tool_calls_only() {
let msg = Message {
role: Role::Assistant,
content: vec![ContentPart::ToolCall(ToolCall::new(
"call_1",
"get_weather",
serde_json::json!({"city": "SF"}),
))],
name: None,
tool_call_id: None,
};
let translated = translate_messages(&[msg]);
assert_eq!(translated.len(), 1);
assert_eq!(translated[0].role, "assistant");
assert!(translated[0].content.is_none());
let tool_calls = translated[0].tool_calls.as_ref().unwrap();
assert_eq!(tool_calls.len(), 1);
assert_eq!(tool_calls[0].id, "call_1");
assert_eq!(tool_calls[0].kind, "function");
assert_eq!(tool_calls[0].function.name, "get_weather");
assert_eq!(tool_calls[0].function.arguments, r#"{"city":"SF"}"#);
}
#[test]
fn translate_assistant_message_with_text_and_tool_calls() {
let msg = Message {
role: Role::Assistant,
content: vec![
ContentPart::text("Let me check the weather"),
ContentPart::ToolCall(ToolCall::new(
"call_2",
"get_weather",
serde_json::json!({"city": "NYC"}),
)),
],
name: None,
tool_call_id: None,
};
let translated = translate_messages(&[msg]);
assert_eq!(
translated[0].content.as_deref(),
Some("Let me check the weather")
);
let tool_calls = translated[0].tool_calls.as_ref().unwrap();
assert_eq!(tool_calls.len(), 1);
assert_eq!(tool_calls[0].function.name, "get_weather");
}
#[test]
fn translate_assistant_message_with_raw_arguments() {
let mut tc = ToolCall::new("call_3", "search", serde_json::json!({"q": "rust"}));
tc.raw_arguments = Some(r#"{"q": "rust"}"#.to_string());
let msg = Message {
role: Role::Assistant,
content: vec![ContentPart::ToolCall(tc)],
name: None,
tool_call_id: None,
};
let translated = translate_messages(&[msg]);
let tool_calls = translated[0].tool_calls.as_ref().unwrap();
// Should prefer raw_arguments over serializing arguments
assert_eq!(tool_calls[0].function.arguments, r#"{"q": "rust"}"#);
}
#[test]
fn translate_tool_message_has_tool_call_id() {
let msg = Message::tool_result(
"call_1",
serde_json::Value::String("72F and sunny".into()),
false,
);
let translated = translate_messages(&[msg]);
assert_eq!(translated[0].role, "tool");
assert_eq!(translated[0].tool_call_id.as_deref(), Some("call_1"));
assert!(translated[0].tool_calls.is_none());
}
#[test]
fn translate_user_message_has_no_tool_calls() {
let msg = Message::user("Hello");
let translated = translate_messages(&[msg]);
assert_eq!(translated[0].role, "user");
assert_eq!(translated[0].content.as_deref(), Some("Hello"));
assert!(translated[0].tool_calls.is_none());
}
#[test]
fn assistant_tool_calls_serialize_correctly() {
let msg = Message {
role: Role::Assistant,
content: vec![ContentPart::ToolCall(ToolCall::new(
"call_1",
"get_weather",
serde_json::json!({"city": "SF"}),
))],
name: None,
tool_call_id: None,
};
let translated = translate_messages(&[msg]);
let json = serde_json::to_value(&translated[0]).unwrap();
assert!(json.get("content").is_none());
assert!(json.get("tool_call_id").is_none());
let tool_calls = json["tool_calls"].as_array().unwrap();
assert_eq!(tool_calls.len(), 1);
assert_eq!(tool_calls[0]["type"], "function");
assert_eq!(tool_calls[0]["id"], "call_1");
assert_eq!(tool_calls[0]["function"]["name"], "get_weather");
}
#[test]
fn audio_content_produces_text_fallback() {
let msg = Message {
role: Role::User,
content: vec![ContentPart::Audio(AudioData {
url: Some("https://example.com/audio.wav".to_string()),
data: None,
media_type: None,
})],
name: None,
tool_call_id: None,
};
let translated = translate_messages(&[msg]);
assert_eq!(
translated[0].content.as_deref(),
Some("[Audio content not supported by this provider]")
);
}
#[test]
fn document_content_produces_text_fallback_with_filename() {
let msg = Message {
role: Role::User,
content: vec![ContentPart::Document(DocumentData {
url: Some("https://example.com/doc.pdf".to_string()),
data: None,
media_type: None,
file_name: Some("report.pdf".to_string()),
})],
name: None,
tool_call_id: None,
};
let translated = translate_messages(&[msg]);
assert_eq!(
translated[0].content.as_deref(),
Some("[Document 'report.pdf': content type not supported by this provider]")
);
}
#[test]
fn document_content_produces_text_fallback_without_filename() {
let msg = Message {
role: Role::User,
content: vec![ContentPart::Document(DocumentData {
url: None,
data: Some(vec![1, 2, 3]),
media_type: None,
file_name: None,
})],
name: None,
tool_call_id: None,
};
let translated = translate_messages(&[msg]);
assert_eq!(
translated[0].content.as_deref(),
Some("[Document content not supported by this provider]")
);
}
#[test]
fn mixed_text_and_audio_content_concatenates() {
let msg = Message {
role: Role::User,
content: vec![
ContentPart::text("Check this: "),
ContentPart::Audio(AudioData {
url: None,
data: Some(vec![1, 2]),
media_type: None,
}),
],
name: None,
tool_call_id: None,
};
let translated = translate_messages(&[msg]);
assert_eq!(
translated[0].content.as_deref(),
Some("Check this: [Audio content not supported by this provider]")
);
}
}

View file

@ -0,0 +1,143 @@
//! Serde types mirroring the OpenAI Chat Completions wire shapes.
#[derive(serde::Serialize)]
pub(super) struct ApiRequest {
pub model: String,
pub messages: Vec<ChatMessage>,
#[serde(skip_serializing_if = "Option::is_none")]
pub temperature: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub max_tokens: Option<i64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub top_p: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub stop: Option<Vec<String>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub tools: Option<Vec<serde_json::Value>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub tool_choice: Option<serde_json::Value>,
#[serde(skip_serializing_if = "Option::is_none")]
pub response_format: Option<serde_json::Value>,
#[serde(skip_serializing_if = "Option::is_none")]
pub stream: Option<bool>,
}
#[derive(serde::Serialize)]
pub(super) struct ChatMessage {
pub role: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub content: Option<String>,
/// Reasoning/thinking content echoed back for providers that require it
/// (Kimi).
#[serde(skip_serializing_if = "Option::is_none")]
pub reasoning_content: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub tool_call_id: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub tool_calls: Option<Vec<ChatToolCall>>,
}
#[derive(serde::Serialize)]
pub(super) struct ChatToolCall {
pub id: String,
#[serde(rename = "type")]
pub kind: String,
pub function: ChatFunction,
}
#[derive(serde::Serialize)]
pub(super) struct ChatFunction {
pub name: String,
pub arguments: String,
}
// --- Response types (non-streaming) ---
#[derive(serde::Deserialize)]
pub(super) struct ApiResponse {
pub id: String,
pub model: String,
pub choices: Vec<ApiChoice>,
pub usage: Option<ApiUsage>,
}
#[derive(serde::Deserialize)]
pub(super) struct ApiChoice {
pub message: ApiChoiceMessage,
pub finish_reason: Option<String>,
}
#[derive(serde::Deserialize)]
pub(super) struct ApiChoiceMessage {
pub content: Option<String>,
pub reasoning_content: Option<String>,
pub tool_calls: Option<Vec<ApiToolCall>>,
}
#[derive(serde::Deserialize)]
pub(super) struct ApiToolCall {
pub id: String,
pub function: ApiFunction,
}
#[derive(serde::Deserialize)]
pub(super) struct ApiFunction {
pub name: String,
pub arguments: String,
}
#[derive(serde::Deserialize)]
#[allow(
clippy::struct_field_names,
reason = "Field names mirror the provider API payload."
)]
pub(super) struct ApiUsage {
pub prompt_tokens: i64,
pub completion_tokens: i64,
}
// --- Streaming response types ---
#[derive(serde::Deserialize)]
pub(super) struct StreamChunk {
pub id: Option<String>,
pub model: Option<String>,
pub choices: Option<Vec<StreamChoice>>,
pub usage: Option<ApiUsage>,
}
#[derive(serde::Deserialize)]
pub(super) struct StreamChoice {
pub delta: Option<StreamDelta>,
pub finish_reason: Option<String>,
}
#[derive(serde::Deserialize)]
pub(super) struct StreamDelta {
pub content: Option<String>,
/// Reasoning/thinking content (used by Kimi and other reasoning models).
pub reasoning_content: Option<String>,
pub tool_calls: Option<Vec<StreamToolCall>>,
}
#[derive(serde::Deserialize)]
pub(super) struct StreamToolCall {
pub index: usize,
pub id: Option<String>,
pub function: Option<StreamFunction>,
}
#[derive(serde::Deserialize)]
pub(super) struct StreamFunction {
pub name: Option<String>,
pub arguments: Option<String>,
}
// --- Accumulated tool call state for streaming ---
pub(super) struct AccumulatedToolCall {
pub id: String,
pub name: String,
pub arguments: String,
pub started: bool,
}

View file

@ -1,5 +1,6 @@
pub mod adapter_registry;
pub mod client;
mod codec;
pub mod error;
pub mod generate;
pub mod middleware;

File diff suppressed because it is too large Load diff