Commit graph

31 commits

Author SHA1 Message Date
devin-ai-integration[bot]
f97d3df6de
fix(cost): bill custom priced deployments at their own rates over the upstream x-litellm-response-cost header (#46120)
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-13 00:58:03 +00:00
devin-ai-integration[bot]
e16d456291
test(integration): isolate every multi-worker integration suite by default (#46027)
* test(integration): isolate every multi-worker integration suite by default

Per-worker database, Redis, upstream and proxy were opt-in for management,
accounting, providers and extensions. mcp, security and cost ran 4 to 8
xdist workers against one shared proxy, database and Redis, and those suites
hold most of the integration flakes CircleCI records. Every run with more than
one worker now builds a private stack per worker, run_integration.sh no longer
starts or exports the shared proxy, upstream, database or Redis in that mode,
and each worker's scripted upstream runs in its own process instead of a
thread in the pytest controller.

Worker gateways get what the shared proxy used to provide: a peer for
management and mcp, the scripted cost map for cost, stdio MCP servers, and
coverage for mcp. mcp coverage now measures lines through sys.monitoring,
because branch measurement on CPython 3.12 uses the trace function and that
made a covered proxy take about 4x as long to become ready, which is why the
shared mcp proxy kept missing its 90 s readiness wait. Routing parity
snapshots now also read the worker databases, since its cost and mcp runs use
several workers. The security per-worker fixture is redundant and removed

* test(integration): give the nested probe root the files a worker gateway needs

Two worker probes now run isolated, so the nested controller boots a real per-worker upstream and proxy from the probe root. That upstream imports the fake OpenAI server and the cost map modules, and the proxy reads proxy_config.yaml, none of which the probe copied. The nested run also drops INTEGRATION_COVERAGE_FILE so it never writes into an outer coverage file

* test(integration): budget nested multi-worker runs for their worker stack boots

A nested run.py database run with more than one worker now boots a private database, Redis, upstream and proxy per worker, one after another, before any case runs. The fixed 60 s subprocess timeout left no room for that, so the 2 worker probes sat at 57-60 s on CircleCI. The subprocess budget is now the old 60 s plus 30 s per worker stack, and the outer pytest timeout keeps its previous 30 s of slack on top. 30 s is the slowest worker proxy readiness recorded in startup.json on the integration-cost job of workflow e09b108b (21.6-30.3 s on 4 CPUs, 2026-10-12)

---------

Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
2026-10-12 03:46:06 -07:00
devin-ai-integration[bot]
3090f4b23e
fix(ci): isolate integration workers and repair test fixtures (#45863)
* test(integration): wait for customer spend rows and use owned OpenRouter cost

* ci: isolate integration workers and clean up resources after crashes

* docs: describe isolated extensions workers and shard census

* fix(ci): restore integration service and fixture guarantees

* fix(integration): preserve replay IDs and native route fixtures

* fix(tests): scope integration harness imports to each unit case

* fix(ci): preserve fixtures and wait for owned workers

* fix(ci): isolate nested runs and telemetry cases

* fix(ci): make worker readiness and fault tests deterministic

* chore(ci): drop MCP fixture repair already covered by main

* chore(ci): keep the local catalog fix from main

---------

Co-authored-by: yuneng <yuneng@berri.ai>
2026-10-11 01:56:02 -07:00
devin-ai-integration[bot]
1dbde0845e
test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758)
* test: move guardrails, image, audio, batches, search legacy tests to unit

* test: restore the custom guardrail and remaining chunk 1 unit rows

* test: move the ocr matrix to e2e

* test: drop comments from moved tests and fix vertex ocr credentials

* test: move litellm utils and unified google legacy tests to their tiers

* test: move logging callback and pass-through unit legacy tests to their tiers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: move store_model_in_db legacy tests and start the admin ui migration

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: finish the proxy job rows and drop their CircleCI jobs

Moves the remaining multi-instance, openai endpoint, otel, pass-through and
top-level proxy tests into tests/integration with scripted upstreams, rewrites
the weak replacements to assert the outbound wire and exact responses, and
removes the build_and_test job group this PR owns from CircleCI

* test: tighten the key generation and error log assertions, rename migrated unit files to their mapped names

* test: shard the pagerduty unit tests and record the rust harness file on the coverage allowlist

* test: drive the SSO sign-in tests through a scripted IdP and isolate the cost center stand-in

* test: type the e2e beta header, prompt cache and secret manager values

* test: assert the missing model error message on the OpenAI routes

* test: drop the deleted otel base class import and restore mcp redaction and exact langfuse usage checks

* test: make the langfuse fixture directory a package for the layout check

* test: restore collected no-match cases, real langfuse and gcs payload checks, and gate the aws secret lane

* test: keep the ocr e2e pdf fixture next to the test

* test: put meta below covers on the streaming tool search e2e

* test: run the OCR matrix against a scripted upstream and fix two runner-only e2e failures

* test: count only OCR calls in the OCR matrix recorder

* test: pin OCR parse bodies and billed cost, group new e2e cases into classes

* ci: restore the python image anchor dropped in the main sync

---------

Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-11 00:17:21 -07:00
devin-ai-integration[bot]
f6bb87739d
test(integration): move the openai cost tracking cases from cost_tracking_cases.json to CostTrackingTestCase literals (#45879)
* test(integration): move the openai cost tracking cases from cost_tracking_cases.json to CostTrackingTestCase literals

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): move the remaining cost tracking cases, the cost map and the batch/realtime cases from cost_tracking_cases.json to Python literals (#45885)

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-10 23:29:23 +00:00
devin-ai-integration[bot]
a2e334febe
test: migrate local_testing SDK and provider tests out of build_and_test (#45760)
* test: migrate local_testing SDK and provider tests out of build_and_test

* test: cover sagemaker region, hf template and nova paths end to end

Region rows now go through litellm.completion on the sagemaker/ handler that
the legacy tests exercised, the prompt template test mocks the Hugging Face
tokenizer config instead of switching to custom_prompt_dict, streaming tests
assert role, finish reason and body, and Nova single turn and streaming run
through litellm.completion with a recorded request body

* test: tighten migrated integration tests for local OpenAI routes, timeouts and air-gapped boot

Split the catch-all providers migration file by behavior. The fake endpoint
route tests now hit the owned upstream directly, as the legacy harness tests
did. LM Studio asserts the outbound key, path and stripped model. The embedding
timeout test asserts the proxy maps a deployment timeout to a 408 and forwards
the read timeout to the provider client, instead of timing out the test client

The Azure streaming tool-call test also asserts the second identical stream is
served from the Redis cache with the same reassembled tool call

The Docker network test always skipped in CI because the security job has no
litellm/litellm image and no egress to pull one, and it tested a published image
rather than the checkout. It now boots the checkout proxy with every outbound
HTTP connection refused and asserts readiness, the bundled cost map fallback and
that the cost map host is the only egress attempted

* test: tighten the migrated e2e rows for local_testing SDK and provider tests

Drop the duplicated Anthropic covers marker and the Anthropic cell wrongly claimed by
the Bedrock and Vertex cache tests. Keep the file-wide Anthropic model as it is on main.
Send the 2023-06-01 version header to Anthropic through the deployment's extra_headers
so the header actually reaches the provider, and mark the creation-token case 1h then
5m like the legacy test did

Reuse the Bedrock file's own registration for the pydantic response test, swap the
hand-built PNGs for the shared cat.jpg fixture, assert the SageMaker Nova image answer
names the cat and that top_logprobs=3 is honored, and register the SageMaker Nova opt-in
markers the same way as the other opt-in markers

* test: restore lost assertions in migrated test_main tests

Restore the role alternation check in test_completion_anthropic_hanging, the
embedding and streaming response-header checks, the logit_bias drop check and
the gemini system and safety settings checks. Add an azure 429 retry-after
test for test_exception_with_headers, trigger the router fallback with
mock_timeout like the legacy test, and stop the mock timeout tests sleeping 3s

* test: restore lost legacy assertions in migrated vertex ai unit tests

* test: tighten migrated cost, streaming and embedding tests and scope the hosted mock guard

* test: strengthen migrated anthropic, azure, bedrock and cohere provider tests

Assert full request bodies and usage instead of presence checks, restore the
dummy-tool and custom_llm_provider assertions the legacy tests made, add the
async and per-model variants the legacy parametrization covered, split
streamed tool arguments across deltas, and move the Azure chat streaming
tests to the azure.py mirror file

* test: tighten migrated llms provider tests and cover legacy intents lost in the move

Strengthen the migrated fireworks, gemini, mistral, together, openai text completion, openrouter, perplexity and openai error tests so they assert what the legacy tests proved (cost, provider tag, usage-only-on-last-chunk, function_call rebuild, reasoning param, sync/async/stream error fields). Replace the huggingface and jina tests that doubled litellm code with respx end-to-end tests. Add the tests the pre-existing mapped tests never covered: cohere 400 mapping, token-array text completion, hosted_vllm/llamafile/lm_studio env api base embeddings, watsonx embedding auth, ollama ssl_verify client and ollama vision prompt

* test: cover residual legacy paths for anthropic tool streams, watsonx async stream, ollama http images and caller async embedding clients

* test: restore single request assertions in the deep_infra and azure_ai embedding tests

* test: drop a no-op streaming cost flag from the openai streaming cost test

* test: cover residual legacy-test lines in caching, main, utils, replicate, vertex and bedrock invoke

* test: cover streamed audio assembly, choice membership, untranslated responses events and the stream log guard

* test: pin cost rates in unit tests, use sdk for cohere e2e, drop vertex llama e2e

* test: send the vertex maps e2e request through the openai sdk

* test: drop the local_testing jobs that select no tests after the migrations

* test: wait for the streamed cache entry and keep custom Mistral rates test-local

* test: keep untouched Mistral test lines as they were

---------

Co-authored-by: yuneng <yuneng@berri.ai>
2026-10-10 23:14:25 +00:00
Mateo Wang
70e6a2ff24
fix(vertex_ai): dial the multi-region Live API host for realtime sessions (#45166)
* fix(vertex_ai): dial the multi-region Live API host for realtime sessions

A realtime deployment with vertex_location us or eu dialed
{location}-aiplatform.googleapis.com, which Vertex does not serve for a
multi-region, so the WebSocket handshake came back 404. The URL builder now
resolves its host through the shared Vertex host resolver, which already
maps us and eu to aiplatform.{geo}.rep.googleapis.com and leaves regions
and global unchanged.

* test(vertex_ai): annotate the realtime multi-region URL tests

* fix(vertex_ai): skip the TLS argument when the realtime api_base is plain ws://

The Vertex realtime session and health-check connects passed the shared
TLS context to every websocket dial, so an api_base the transformation
already maps from http:// to ws:// failed with 'ssl argument is
incompatible with a ws:// URI' before any frame was sent. The OpenAI
realtime handler already skips the argument for ws://; the shared helper
now does the same for the Vertex path

* test(integration): cover the Vertex realtime multi-region host through the proxy

Fourteen cells on the scripted Gemini Live upstream: api_base overrides for
us, eu, us-central1 and global complete a turn; malformed locations are
refused before any dial; a missing location defaults to us-central1; the
realtime health check handshakes the override host and reports a malformed
location without dialing; a refused handshake reaches the client and the
next session connects; twelve concurrent sessions across locations each
reach the upstream once; an upstream outage closes every open session and
the proxy recovers

* test(realtime): type the health-check helpers and flatten the burst order

* test(http_handler): type the realtime TLS test signatures
2026-10-07 17:39:38 -07:00
devin-ai-integration[bot]
68c0a972a7
fix(realtime): skip guardrail VAD session.update injection for transcription sessions (#44843)
* fix(realtime): skip guardrail VAD session.update injection for transcription sessions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(realtime): flag transcription sessions from the route intent and backend events only

A client session.update declaring session.type transcription on a voice
session no longer sets the transcription flag, so it cannot switch off the
guardrail's create_response gate or skip the transcript guardrail

* fix(realtime): flag transcription sessions from provider-transformed session events

* test(integration): cover transcription sessions skipping the VAD auto-response injection

Adds the realtime transcript guardrail audit cells: transcription sessions on all three
realtime routes, the OpenAI SDK, the beta protocol, the whisper default deployment, the Azure
GA path over a TLS scripted upstream, and Meta Muse push-to-talk sessions keep the client's
session.update verbatim and get their transcript, while voice sessions keep the injected
create_response gate and a client-declared transcription type no longer bypasses it. Sad,
edge, and chaos cells cover malformed session fields, duplicate and older backend session
events, unauthenticated upgrades, repeated sessions, an upstream outage under open sessions,
and a worker kill with a proxy restart.

The scripted upstream now answers session.update the way the vendor does (session.updated,
or the missing turn_detection.type and session-type errors), records every websocket frame,
serves the Azure and Muse realtime paths, and can run over TLS from an owned upstream.

---------

Co-authored-by: gabriele <gabriele@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-06 22:42:00 -07:00
devin-ai-integration[bot]
a45c6b8f65
test(integration): keep scripted upstream connections open past the proxy's keepalive (#44695)
* test(integration): re-pin claude-sonnet-5 stream recount tokens and outlast proxy keepalive in scripted upstream

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): derive scripted upstream keepalive from AIOHTTP_KEEPALIVE_TIMEOUT

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): hardcode scripted upstream keepalive to avoid importing litellm

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-06 03:13:06 +00:00
michelligabriele
5dd77eb8ca
fix(openai/realtime): drop model from upstream URL for intent=transcription (#43854)
* fix(openai/realtime): drop model from upstream URL for intent=transcription

* fix(openai/realtime): keep model on the upstream URL for non-OpenAI hosts

A custom api_base on the openai provider can be a gateway that routes on
the model query param, so the transcription model drop now applies only
to OpenAI's own hosts. xAI's URL is unchanged again

* test(integration): audit realtime transcription upstream URL on a custom api_base

Twenty-two integration cells cover transcription and conversation sessions on every realtime route, the OpenAI SDK sync and async clients, malformed and duplicated query params, unauthenticated and refused upgrades, an unknown model, idempotent spend logging, a twenty-session burst, an upstream outage mid burst, a worker kill, and a proxy restart, each asserting the exact query pairs the upstream received. The scripted upstream now records every websocket upgrade as an observation

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-05 19:14:56 +00:00
devin-ai-integration[bot]
6d8434f940
fix(proxy): return 4xx instead of 500 for missing required params, invalid pagination and unknown ids (#43787)
* fix(proxy): return 400 instead of 500 for missing required params and invalid pagination

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: run search_endpoints tests in proxy-endpoints shard

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): return 4xx for missing required params across all LLM routes and propagate provider status on lookups

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(llm_http_handler): keep provider error text when re-raising mapped errors

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): allow promptless image edits and default search models

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): default missing image edit image to None

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(proxy): build image edit defaults without mutating request data

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): keep image edit defaults within type-discipline budget and give request mocks a scope

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): inject a fake router for the search default model test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(llms): cover provider error status on vector store and file lookup handlers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(llms): keep the lookup handler raise block to a single statement

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(llms): cover provider error status on eval, eval run, skill and vector store file content lookups

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover missing required body params and provider lookup status codes

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: run tests/unit/proxy/search_endpoints in the proxy-endpoints shard

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): bind spend-row request id with partial to satisfy B023

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): only reject non-positive page_size on vector store list

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): remove unreachable fine-tuning body validation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover streaming anthropic messages reaching the upstream

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): count only provider calls when asserting missing params never reach the upstream

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): preserve merge-base request compatibility

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): preserve interaction completion model defaults

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): retry model read-through before rejecting params a DB-only deployment may default

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
2026-10-02 22:48:53 -07:00
devin-ai-integration[bot]
8596fe954d
feat(interactions): durable cross-pod settlement for background interaction billing (#41955)
* feat(interactions): durable cross-pod settlement for background interaction billing

Background interaction billing lived only in the creating replica's memory, so a
DELETE routed to another replica, or a restart of the creating one, never billed
the completed provider work and the budget reservation was refunded at the poll
timeout. The create now registers the billing context in a settlement store
before returning, the proxy installs a Prisma-backed store at boot
(LiteLLM_BackgroundInteractionSettlement, schema-only migration), any replica
claims the row once through a conditional update before billing or releasing,
startup resumes every unclaimed row with its remaining timeout, and a give-up
records an unsettled outcome instead of silently reconciling to zero. The SDK
keeps an in-memory store and behaves as before.

* fix(interactions): survive a settlement install failure at boot and stop carrying request headers

* fix(interactions): drop the stored request context once a settlement row is settled

* fix(interactions): bill the completed response a poll already saw when its claim only answers at the deadline

* fix(interactions): carry a missing model through the settlement context for agent-only background creates

An interaction created with an agent and no model reaches the poll with no model name, exactly as on main. The settlement context now stores that None instead of rejecting the create, which answered the client with a 500 after the provider had already accepted it.

* fix(interactions): leave an unfetchable background interaction to its creating poll when a delete lands elsewhere

The remote pre-delete path fetches with only the delete's credentials, so a fetch it cannot make says nothing about the interaction. It used to claim the settlement row and release the reservation anyway, which stopped the creating replica's poll and lost the bill when the delete then failed the same way. It now returns without claiming; the in-process path keeps releasing on an unfetchable state, since its context carries the create's own credentials.

* fix(interactions): fail a cross-replica delete when its pre-delete fetch fails so the creating poll keeps the bill

* fix(interactions): keep the stored settlement gate when registration raises after landing, and fail resumed-poll deletes closed

A registration that raised after its row committed moved the poll to a private in-memory gate, so the creating worker billed while the stored row stayed unclaimed for another replica's delete or the next boot to bill again. The row is now read back once and, when it landed, the poll claims through it like every other settler.

A worker that resumed the poll after a restart is not the creator, so its delete on a failed pre-delete fetch now fails with the fetch's error instead of releasing and deleting. After a fleet restart every worker holds resumed polls, which left the fail-closed path applying nowhere.

* fix(interactions): settle an unverified registration through the durable claim

A create whose settlement-store write raised no longer bills through a
private in-memory gate that a later boot's resume cannot see. The claim
asks the durable store first and falls back to the local gate only when
the store answers that no row exists, and a missing settlement table reads
as no rows so a replica without the migration still settles in process.

* test(proxy): keep the settlement test where the proxy-infra shard collects it

The merge of main moved test_background_interaction_settlement.py under
tests/unit/proxy/spend_tracking, but the proxy-db shards claim tests/unit/proxy
files one by one in .circleci/scripts/unit_selection.sh, so no CI shard ran it
and codecov/patch dropped. tests/test_litellm/proxy/spend_tracking is collected
whole by the proxy-infra shard, which is where the test ran before the merge.

* fix(interactions): raise on a non-2xx Gemini interaction fetch

AsyncHTTPHandler.get never raises for status and the Gemini GET transform
only raised when the body was not JSON, so a 500 or 404 carrying Gemini's
JSON error body parsed as an interaction with no status. A delete on a
replica other than the creator then claimed the settlement as released and
forwarded the delete instead of failing closed, and the bill was lost. The
transform now raises GeminiError with the vendor's status, as the delete
transform already does; the in-process poll already retries a fetch that
raises

* test(integration): audit durable background interaction settlement across replicas

Twenty-six deterministic cells drive a one-worker creator and a two-worker
settler against an owned scripted Gemini upstream: cross-replica deletes
bill once, failed and cancelled interactions release, a later replica
resumes unclaimed rows, custom deployment pricing bills at the deployment
rate, a fetch the settler cannot make fails the delete closed, odd ids are
refused, a missing settlement table keeps in-process billing, polling
disabled registers nothing, the budget reservation is released by the
settler, an upstream outage mid-burst fails closed and recovers, killed
workers hand their polls to the respawned ones, and concurrent deletes on a
slow upstream settle exactly once. The support upstream gains a scripted
interaction store with per-id GET status and delay, and the process helper
gains an owned upstream a test can stop and restart

* test(integration): refuse a repeated delete in the scripted upstream and pin the settlement budget below one estimate

* chore(ui): regenerate dashboard API types after merging main

* test(integration): accept the 422 budget refusal and a respawned worker's resume

The budget cell pinned a 400 that the proxy stopped answering when budget refusals moved to 422, so it now asserts the status and the budget_exceeded error type the sibling budget tests pin. The later-booting replica cell accepts a claimer that is any worker started after the creates, since uvicorn's supervisor can respawn the creator's worker under load and the respawned worker's boot resume claims the rows by design; the single spend row check is unchanged

* test(integration): delete the pinned key's interaction with a second key

A key whose budget is filled by its own reservation is refused on every route, the DELETE included, so the cell now asserts that 422 and sends the delete with a second key, which is what the reservation release on another replica needs in order to be observable at all

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-02 21:08:38 -07:00
devin-ai-integration[bot]
8b4de39ad7
fix(bedrock): surface unrecognized converse-stream event frames instead of an empty turn (#44112)
* fix(bedrock): surface unrecognized converse-stream event frames instead of an empty turn

* fix(bedrock): track distinct unknown stream event types and cover the stream error builder

* test(bedrock): audit converse-stream event frames on the proxy and the sync decoder

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-02 16:06:26 -07:00
devin-ai-integration[bot]
e450f2d54c
fix(router): serve Responses turns from a sibling when the encrypted content origin has no boundary peer (#43015)
* test(integration): reproduce encrypted_content_affinity 503 when origin has no boundary peer

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): keep encrypted content affinity turn one out of the response cache

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(router): degrade encrypted_content_affinity when the origin has no encryption-boundary peer

A routed-group candidate that is currently unavailable and shares its
(api_base, api_key) with no healthy deployment used to raise a proxy-level
503/429 from _unavailable_origin_error, even though healthy siblings in the
same model group could still serve the turn. Strip the encrypted reasoning
and dispatch to the healthy pool instead, matching the existing cross-group
behavior, and log a warning naming the origin model_id and routed group

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(router): bound the degraded-affinity log marker and assert the strip on the wire

Address review findings: restore num_retries alongside
optional_pre_call_checks in the integration test teardown, record scenario
request bodies on the scripted upstream so the tests can assert no encrypted
reasoning reaches the sibling, and truncate the client-supplied model_id in
the degraded-dispatch warning

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(tests): only record JSON bodies on the scripted upstream

Multipart uploads to scripted POST routes have no JSON body, so gate the
observation recording on the request content-type

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(router): make encrypted_content_affinity runtime-toggleable so /config/update can turn it off

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-24 17:31:43 -07:00
devin-ai-integration[bot]
f44052d87b
fix(vector_stores): keep config-defined vector stores listed and read-only (#42574)
* fix(vector_stores): keep config-defined vector stores listed and read-only

Vector stores declared in config.yaml were purged from the in-memory registry by /vector_store/list because the database was treated as the only source of truth. Config-defined stores now carry is_config=True, stay in the list beside database rows, are never overwritten or evicted by database state, and reject /vector_store/new, /vector_store/update and /vector_store/delete with 400. The Admin UI renders them read-only

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): show vector store source and read-only state for config-defined stores

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): audit config-owned vector stores across list, writes, search, authz, peers and redis outage

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): show a visible read-only hint in the config vector store actions menu

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 04:02:16 +00:00
kerry
807541291d test(integration): batch and realtime cost cases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 01:25:28 +00:00
kerry
2edea0be08 test(integration): assert recount pins and unique fixture request ids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 00:36:18 +00:00
kerry
a841750d46 test(integration): proxy behaviour cost cases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 00:19:56 +00:00
kerry
a4488fccca test(integration): add provider wire cost cases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 23:45:51 +00:00
kerry
110d4c2ad1 test(integration): add passthrough cost cases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 22:05:57 +00:00
kerry
c0c5cc84f8 test(integration): audio, image and per-unit cost cases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 21:19:29 +00:00
kerry
e42f3f1562 test: fix cost harness review issues
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 19:10:04 +00:00
kerry
c958f9db7e test: extend cost tracking integration harness
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 18:44:51 +00:00
kerry
ed0c32cdb0 test(integration): drive cost tracking from literal request/response data
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 04:04:40 +00:00
kerry
57d2fefa8d test(integration): derive scripted shapes from litellm provider configs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 03:16:16 +00:00
kerry
6ccba7fdb5 test(integration): drive scripted wires and provider wiring from data
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 01:41:19 +00:00
kerry
77f6166c39 test(integration): fold scenario client into upstream module
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 01:30:24 +00:00
kerry
e52eea84e6 test(integration): serve scripted wires from the shared upstream
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:50:26 +00:00
Yuneng Jiang
fa5d31a837
test(integration): send OpenAI-shaped error types from the fake upstream
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Since #40994 the proxy relays the upstream error body on a 400, so the
public error type is now whatever the upstream sent instead of the
status-derived name. The fake upstream answered every scripted failure
with type api_error, which made the public-error contract in
test_retry_counts_and_public_errors_match_actual_provider_attempts fail
on main. The fake now sends the type a real OpenAI-compatible upstream
sends for the status, so the assertion holds whether the proxy relays
or maps the type
2026-09-16 13:14:47 -07:00
Yuneng Jiang
cfda6dea3e
test: cover persisted updates and warmed authorization policies 2026-09-14 04:20:06 -07:00
Yuneng Jiang
92e0b72e2d
test: add CircleCI integration contract foundation 2026-09-14 03:30:52 -07:00