Commit graph

13 commits

Author SHA1 Message Date
Bryan Helmkamp
cc590f6f97
Merge remote-tracking branch 'origin/main' into fix/modal-provider-catalog
Resolve the model catalog table conflict in docs/public/core-concepts/models.mdx
by keeping both changes: this branch's `kimi` -> `moonshot` provider rename for
the Kimi rows, and main's new DeepSeek V4 rows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 09:32:19 -04:00
Release Repro
76dae568f5
feat(reasoning): expose DeepSeek effort controls 2026-07-31 13:14:11 -04:00
Bryan Helmkamp
24f9ac6bd1
refactor(model): rename Kimi provider to Moonshot 2026-07-31 09:00:52 -04:00
Bryan Helmkamp
d4f619bc2a
fix: clean up inference observability 2026-07-24 22:50:00 -04:00
Bryan Helmkamp
e1d0b1af4f
refactor(llm): make StreamStart a universal liveness edge
`StreamStart` was supposed to mean "the provider is responding", but
each decoder decided for itself when to emit it, so it meant something
different per dialect:

  anthropic       on the `message_start` frame
  bedrock         on the `messageStart` frame
  openai_responses  latched on the first SSE event
  gemini          latched on the first chunk
  openai_compatible  never — Chat Completions has no opening frame

A consumer could not rely on it, which is why the inference-bracket
work keyed its first-output edge on content kind instead.

Ownership moves to the two loops that drive decoders — the shared SSE
loop in `transport.rs` and the AWS event-stream loop in the bedrock
provider — each emitting exactly one `StreamStart` immediately before
handing over the first framed event. The invariant is now structural:
it cannot depend on a dialect having a particular opening frame,
because no decoder is involved in producing it. `StreamDecoder`
documents that decoders must not emit it, and the four that did no
longer do.

Only `openai_compatible` changes observably, gaining the event it never
had; the other three dialects' snapshots are byte-identical, because
their opening frame was already the first framed event. The six
updated `openai_compatible` snapshots each differ by exactly one
leading `stream_start` and nothing else.

Each dialect also gets an explicit `stream_opens_with_stream_start`
assertion. The snapshots already cover this, but a snapshot can be
re-accepted silently, and this is the one event a liveness consumer
needs to hold for every provider.

No behavior change to the agent's inference bracket:
`first_output_kind()` maps `StreamStart` to `None`, so the bracket
still opens on observed content and keeps reporting which kind
arrived. The point of this change is that a content-agnostic edge now
exists at all — the one-shot coverage follow-up needs it, and it is
strictly earlier than first content.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-24 22:05:26 -04:00
Bryan Helmkamp
7044dd57fa
Merge pull request #627 from fabro-sh/fix/openai-compatible-stream-usage
fix(llm): request streaming usage on openai_compatible providers
2026-07-24 17:53:45 -04:00
Bryan Helmkamp
53a92f75f4
test(llm): model streamed usage in OpenAI twin 2026-07-24 17:47:05 -04:00
Release Repro
512ab50f9c
fix(reasoning): align stream and client invariants 2026-07-24 17:42:44 -04:00
Release Repro
4d3de5f564
fix(reasoning): tighten capture normalization 2026-07-24 17:31:14 -04:00
Bryan Helmkamp
0169725b4e
fix(llm): request streaming usage on openai_compatible providers
Chat Completions only emits the trailing usage chunk when the request sets
`stream_options: {"include_usage": true}`. The openai_compatible codec never
sent it, so providers that follow the spec strictly returned no usage at all
on streamed responses. Every message came back with zero tokens, and the
catalog cost estimate multiplied those zeros into $0.

Kimi is the visible case: a run's kimi-k3 stages report 0 tokens and no
dollars, while an openrouter stage in the same run bills normally because
OpenRouter volunteers usage (and an in-band cost) without being asked.

Send the opt-in whenever we stream. Providers that already volunteer usage
accept the field and are unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-24 16:21:00 -04:00
Bryan Helmkamp
e7740b4acb
feat(reasoning): capture provider reasoning in agent.message
Normalize the readable reasoning providers already return into a
canonical `ReasoningOutput` and carry it through the `agent.message`
run event to storage, SSE, and JSONL.

The shape is derived from the final response's canonical message
content rather than stored a second time, so there is no duplicate
source of truth and retried or replaced streaming buffers never
become durable reasoning. OpenAI-compatible `reasoning_details` are
now preserved verbatim as an opaque content part; only known readable
members are normalized out of them, leaving encrypted entries for a
later provider-aware replay phase.

This phase is passive: no request parameters change, no capability
guessing, and no newly observed provider field is replayed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-24 14:21:15 -04:00
Bryan Helmkamp
3c33e0fb69
Merge origin/main (three-layer crate layout) into fix/openrouter-anthropic-prompt-caching
The crate reorganization renamed lib/crates/ to lib/apps|components|foundation/.
Git followed all modified files across the rename; the only conflict was the
newly added codec/cache.rs, now placed at lib/components/fabro-llm/src/codec/cache.rs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 19:30:20 -04:00
Scott Werner
47bc772f7b refactor: organize crates into three layers 2026-07-23 17:59:34 -04:00