The storage branch's three conditions sat inline in `responses_api`, so nothing
proved a foreground create or an already-terminal one stays out of the managed
table. They move into `should_store_background_response`, which the endpoint
calls and the tests exercise across both arms.
Claude-Session: https://claude.ai/code/session_01RHAjRxNhXTpKHeGMZ1nDKi
The seam's type annotation pulled `litellm_enterprise` into this module's import
graph, which check_unsafe_enterprise_import rejects outside a try-except. A
Protocol carrying the one method the seam calls types it without the import and
drops the local one the cast needed too.
Claude-Session: https://claude.ai/code/session_01RHAjRxNhXTpKHeGMZ1nDKi
Two callers only want the provider's own id behind an advertised one, and both
reached past the class to get it out of the decrypt tuple. The poller also typed
its rows as the pydantic projection, which has no `id`, while it is handed a
Prisma row and reads `job.id` six times.
Claude-Session: https://claude.ai/code/session_01RHAjRxNhXTpKHeGMZ1nDKi
The poller narrated its own straightforward behavior in seven multi-paragraph
docstrings, which the repo's comment policy rules out. Each is now the claim a
reader needs to avoid a wrong edit and nothing more. Also drops the deprecated
`Dict` and `Optional` aliases the file still used.
Claude-Session: https://claude.ai/code/session_01RHAjRxNhXTpKHeGMZ1nDKi
The eight claim/release tests reached into `litellm.aget_responses` with
`patch`, which the test-quality gate flags and which couples them to an
import path rather than to the poller's own seam. Each job now carries a
LiteLLM-encoded provider id, so the read routes through the router the
fixture already injects and the assertions run against that mock.
Claude-Session: https://claude.ai/code/session_01RHAjRxNhXTpKHeGMZ1nDKi
The store lived inline in responses_api, so covering it meant patching five
proxy_server globals per test, which the test-quality gate counts as pinning
the test to the wiring rather than the behaviour. It is now a module-level
function that takes the managed-files hook as an argument, and the tests hand
it a fake directly.
The poller tests reach the same fetch through the router fixture that is
already injected, instead of patching litellm.aget_responses.
A response with no model_id now returns early rather than raising into the
caller's except block. Nothing is stored either way and the warning is
unchanged, so only the redundant second log line goes away.
Claude-Session: https://claude.ai/code/session_01RHAjRxNhXTpKHeGMZ1nDKi
model_object_id is documented as "the id returned by the backend API
provider", and that is what batches and fine-tuning jobs store there. The
background responses create stored the advertised id in it instead, which
is encrypted with a fresh nonce on every call, so the row had no stable
handle on the generation it describes.
The cost poller now reads the provider id straight off the row. Rows
written before this still carry the advertised id there, and decrypting is
a no-op on an id that is already the provider's, so both shapes resolve
through the same call.
Claude-Session: https://claude.ai/code/session_01RHAjRxNhXTpKHeGMZ1nDKi
The previous commit stored the finished ResponsesAPIResponse in file_object. That
duplicates content the provider still serves from its own copy, and the usage and
spend it was meant to preserve already land in LiteLLM_SpendLogs on every billed
call regardless of store_prompts_in_spend_logs, which gates only the messages and
response body columns.
The poller now writes status alone, as it did before. The write stays per job
rather than one bulk update so a single failure cannot strand the rest of the
cycle.
Claude-Session: https://claude.ai/code/session_01Hn5E8Jz1LjGLFyiYxBRcBW
Every pod and uvicorn worker schedules its own CheckResponsesCost against the
shared LiteLLM_ManagedObjectTable. The poller selected eligible rows, performed
the billed retrieval, and only then marked them completed in one bulk write, so
two pollers could select the same terminal response and both record a charge
before either completion update landed.
Each row is now claimed with a compare-and-swap on batch_processed before the
read, because the read is what prices the job: aget_responses stamped with the
poll origin writes the spend log itself, so there is no later point at which to
serialize. A row whose read raised, or whose provider status is still
non-terminal, releases its claim so a later cycle retries it rather than
retiring it unbilled. That is the failure #37050 fixed on the batch side.
A pod that dies between winning the claim and billing would otherwise strand the
row: it holds a claim nobody will release and its status never reaches terminal,
so every later cycle re-selects it and loses. The updated_at arm of the claim
takes such a row back after three poll cycles, and since updated_at is @updatedAt
a healthy in-flight claim written moments ago is never stolen.
The poller now also persists the finished response onto its managed row instead
of writing status alone, so the row carries the generation's usage rather than
the stale queued copy stored at create time.
Reuses the existing batch_processed column, so no migration. It already sits on
the shared table defaulted to false and was unused by response rows.
Claude-Session: https://claude.ai/code/session_01Hn5E8Jz1LjGLFyiYxBRcBW
Forwarded client headers on bridged /v1/responses calls were serialized into the
OpenAI JSON body as extra_headers when EXPERIMENTAL_OPENAI_BASE_LLM_HTTP_HANDLER
was set, and OpenAI rejected the request with unknown_parameter. The headers are
already merged into the outgoing HTTP headers, so only set the SDK-style
optional param on the SDK client path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
get_litellm_params() keys such as model_alias_map, completion_call_id and
custom_prompt_dict were missing from all_litellm_params, so when a bridge
forwarded litellm_params as completion kwargs they were swept into extra_body
and OpenAI rejected the call with "Unknown parameter: 'model_alias_map'".
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The PLR0206 fix renamed the property to get_api_version and dropped it.
AzureFoundryModelInfo().api_version does resolve today, so removing it
breaks external callers. Restore it as a zero-arg property delegating to
the staticmethod, which satisfies PLR0206 either way.
