Commit graph

6 commits

Author SHA1 Message Date
mateo-berri
a41ac5c139 fix(interactions): poll queued background creates and drop the poll's deployment identity
Some checks failed
LiteLLM Rust / rustfmt, clippy, test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
`queued` is the Interactions API's not-started-yet status. It was in neither
the pollable set nor the terminal one, so a create returning it got no poll
task, counted as a response with nothing to charge for, and released its
budget reservation: billed nowhere, alerting nobody.

Poll it alongside `in_progress`, and pin the union of the pollable and
terminal sets against the generated spec enum so a status Google adds later
fails CI rather than shipping another unbilled path. A give-up on a status in
neither set now names the status and logs at error, instead of the warning
that reads as an interaction merely still running.

Also drop `model_id` and `litellm_model_name` from the settled body next to
the foreign `response_cost` already dropped there. All three come from the
poll's own throwaway client call, and left in place the two identity fields
overwrite the create's real deployment in the payload every logging
integration reads.

Rewrites the callback's per-status test to assert the observable outcome
(reservation held vs released) across all eight statuses rather than
comparing the gate to the function it delegates to, and pins the shipped
5-10-20-40-60 poll backoff and its timeout cutoff.
2026-08-22 17:35:19 -07:00
mateo-berri
8b566a7f0a fix(interactions): stop two settlement paths from pinning the budget reservation
Both leave a background interaction's pre-call reservation open, so the
serving process keeps refusing traffic on the key at the estimated cost
while its recorded spend stays near zero.

A raise from the completion event propagated out with the settlement gate
already claimed, and nothing retries a claim that is set, so the reservation
was never released. Billing now releases it on the way out.

`requires_action` was missing from the terminal set. It is terminal for the
interaction it names: the API has no operation that resumes one, and a caller
answers a tool request by creating a new interaction whose
`previous_interaction_id` points at it. A function-calling background create
that stopped there was polled until the 3600s timeout, losing the tokens it
had already spent producing the tool request and holding its reservation open
for that whole window.
2026-08-22 15:28:12 -07:00
mateo-berri
6befeb8a17 fix(interactions): stop the cost poll loop instead of spinning on a non-positive interval 2026-08-22 14:48:32 -07:00
mateo-berri
194dca7dd0 fix(interactions): settle pending background interaction billing before delete 2026-07-15 07:15:50 -07:00
mateo-berri
bbb1565772 fix(interactions): hold budget reservation open until background interaction completes 2026-07-15 06:32:16 -07:00
mateo-berri
59d4e52a3d fix(interactions): bill background interactions once completed via cost polling 2026-07-14 20:05:33 -07:00