Commit graph

3 commits

Author SHA1 Message Date
Bryan Helmkamp
d4f619bc2a
fix: clean up inference observability 2026-07-24 22:50:00 -04:00
Bryan Helmkamp
e1d0b1af4f
refactor(llm): make StreamStart a universal liveness edge
`StreamStart` was supposed to mean "the provider is responding", but
each decoder decided for itself when to emit it, so it meant something
different per dialect:

  anthropic       on the `message_start` frame
  bedrock         on the `messageStart` frame
  openai_responses  latched on the first SSE event
  gemini          latched on the first chunk
  openai_compatible  never — Chat Completions has no opening frame

A consumer could not rely on it, which is why the inference-bracket
work keyed its first-output edge on content kind instead.

Ownership moves to the two loops that drive decoders — the shared SSE
loop in `transport.rs` and the AWS event-stream loop in the bedrock
provider — each emitting exactly one `StreamStart` immediately before
handing over the first framed event. The invariant is now structural:
it cannot depend on a dialect having a particular opening frame,
because no decoder is involved in producing it. `StreamDecoder`
documents that decoders must not emit it, and the four that did no
longer do.

Only `openai_compatible` changes observably, gaining the event it never
had; the other three dialects' snapshots are byte-identical, because
their opening frame was already the first framed event. The six
updated `openai_compatible` snapshots each differ by exactly one
leading `stream_start` and nothing else.

Each dialect also gets an explicit `stream_opens_with_stream_start`
assertion. The snapshots already cover this, but a snapshot can be
re-accepted silently, and this is the one event a liveness consumer
needs to hold for every provider.

No behavior change to the agent's inference bracket:
`first_output_kind()` maps `StreamStart` to `None`, so the bracket
still opens on observed content and keeps reporting which kind
arrived. The point of this change is that a content-agnostic edge now
exists at all — the one-shot coverage follow-up needs it, and it is
strictly earlier than first content.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-24 22:05:26 -04:00
Scott Werner
47bc772f7b refactor: organize crates into three layers 2026-07-23 17:59:34 -04:00
Renamed from lib/crates/fabro-llm/tests/it/wire/gemini.rs (Browse further)