Two callers only want the provider's own id behind an advertised one, and both
reached past the class to get it out of the decrypt tuple. The poller also typed
its rows as the pydantic projection, which has no `id`, while it is handed a
Prisma row and reads `job.id` six times.
Claude-Session: https://claude.ai/code/session_01RHAjRxNhXTpKHeGMZ1nDKi
The store lived inline in responses_api, so covering it meant patching five
proxy_server globals per test, which the test-quality gate counts as pinning
the test to the wiring rather than the behaviour. It is now a module-level
function that takes the managed-files hook as an argument, and the tests hand
it a fake directly.
The poller tests reach the same fetch through the router fixture that is
already injected, instead of patching litellm.aget_responses.
A response with no model_id now returns early rather than raising into the
caller's except block. Nothing is stored either way and the warning is
unchanged, so only the redundant second log line goes away.
Claude-Session: https://claude.ai/code/session_01RHAjRxNhXTpKHeGMZ1nDKi
model_object_id is documented as "the id returned by the backend API
provider", and that is what batches and fine-tuning jobs store there. The
background responses create stored the advertised id in it instead, which
is encrypted with a fresh nonce on every call, so the row had no stable
handle on the generation it describes.
The cost poller now reads the provider id straight off the row. Rows
written before this still carry the advertised id there, and decrypting is
a no-op on an id that is already the provider's, so both shapes resolve
through the same call.
Claude-Session: https://claude.ai/code/session_01RHAjRxNhXTpKHeGMZ1nDKi
Every pod and uvicorn worker schedules its own CheckResponsesCost against the
shared LiteLLM_ManagedObjectTable. The poller selected eligible rows, performed
the billed retrieval, and only then marked them completed in one bulk write, so
two pollers could select the same terminal response and both record a charge
before either completion update landed.
Each row is now claimed with a compare-and-swap on batch_processed before the
read, because the read is what prices the job: aget_responses stamped with the
poll origin writes the spend log itself, so there is no later point at which to
serialize. A row whose read raised, or whose provider status is still
non-terminal, releases its claim so a later cycle retries it rather than
retiring it unbilled. That is the failure #37050 fixed on the batch side.
A pod that dies between winning the claim and billing would otherwise strand the
row: it holds a claim nobody will release and its status never reaches terminal,
so every later cycle re-selects it and loses. The updated_at arm of the claim
takes such a row back after three poll cycles, and since updated_at is @updatedAt
a healthy in-flight claim written moments ago is never stolen.
The poller now also persists the finished response onto its managed row instead
of writing status alone, so the row carries the generation's usage rather than
the stale queued copy stored at create time.
Reuses the existing batch_processed column, so no migration. It already sits on
the shared table defaulted to false and was unused by response rows.
Claude-Session: https://claude.ai/code/session_01Hn5E8Jz1LjGLFyiYxBRcBW
Forwarded client headers on bridged /v1/responses calls were serialized into the
OpenAI JSON body as extra_headers when EXPERIMENTAL_OPENAI_BASE_LLM_HTTP_HANDLER
was set, and OpenAI rejected the request with unknown_parameter. The headers are
already merged into the outgoing HTTP headers, so only set the SDK-style
optional param on the SDK client path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
get_litellm_params() keys such as model_alias_map, completion_call_id and
custom_prompt_dict were missing from all_litellm_params, so when a bridge
forwarded litellm_params as completion kwargs they were swept into extra_body
and OpenAI rejected the call with "Unknown parameter: 'model_alias_map'".
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Prompt Security reports a queued sanitization job as status "created" before
it moves to "in progress" and "done". The poller treated anything other than
those two known strings as an error and returned HTTP 500 on the first poll, so
every image or file request through the guardrail failed while the vendor job was
still queued.
Only "done" is terminal now. Every other status is logged and polled again until
max_poll_attempts or the outer file_sanitization_timeout, after which the existing
fail-open or fail-closed (408) policy applies.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Releasing the slot unconditionally from the route raced the limiter's own
success handler on the logging worker: both could read the same stashed
acquisition before either cleared it, and under the integer in-memory
fallback that double-decrements the counter. The route now releases only on
exits without a success callback (pre-call rejection, pre-call cancellation,
and Phase 2 exits without the success stamp), matching the HTTP disconnect
path's ownership rule.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Pre-call limiters reject before a deployment is attached to request_data, so
the failure hook could not resolve api_provider for router aliases and emitted
api_provider="None" on litellm_proxy_failed_requests_metric_total and
litellm_proxy_total_requests_metric_total. Fall back to the provider the
limiter already resolved onto RateLimitError.llm_provider, keeping request
data as the first source and ignoring the proxy placeholder.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
