* chore(github_copilot): sync model catalog with Copilot's served models
* test(github_copilot): assert catalog invariants instead of pinning vendor facts
* fix(github_copilot): keep thinking capabilities on served Claude catalog rows
* fix(github_copilot): keep supports_reasoning on catalog rows that shadow the family fallback
* fix(model-catalog): preserve merged Copilot Haiku metadata
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): list the account's invocable models behind bedrock/* when check_provider_endpoint is on
* fix(bedrock): sign the listing query RFC 3986 style and raise BedrockError on a failed listing
* test(utils): keep test_utils.py at main's formatting, only the Bedrock discovery test is new
* fix(bedrock): chain BedrockModelInfo's constructor so the passthrough config keeps its event parser
* fix(bedrock): list discovered ids without the provider prefix so partial wildcards filter
get_valid_models returns bare vendor ids for every other provider and for the static bedrock catalog, and the proxy's wildcard expansion adds the provider prefix itself. The lister prefixed its ids, so a partial wildcard like bedrock/anthropic.* never matched the proxy's filter and listed uncallable bedrock/anthropic.bedrock/<id> entries
* test(bedrock): integration cells for wildcard discovery through the proxy and the SDK
* fix(proxy): keep a partial wildcard a filter when its deployment repeats the prefix
The wildcard expansion guessed filter-or-alias by whether any provider id carried the prefix. A deployment such as model_name bedrock/anthropic.* over model bedrock/anthropic.* can only ever route names that share the prefix, so when the account has no on-demand anthropic.* id (every Anthropic model behind an inference profile) the guess fell into the alias branch and listed bedrock/anthropic.<every id>, none of them callable. A deployment whose model repeats the suffix now always filters, and lists nothing when nothing matches
* fix(proxy): prefix every discovered id under a custom wildcard prefix that starts a vendor id
* test(bedrock): Final and read-only annotations in the wildcard discovery tests
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(router): honor a per-request fallbacks list on a mid-stream fallback
The completion entrypoints record the per-request fallbacks, context_window_fallbacks, and content_policy_fallbacks in the mid-stream controls carrier the Responses and Messages paths already use, and each completion attempt restores them into the kwargs its stream re-enters the fallback chain with, so a streaming request that dies before its first chunk fails over to the list the request named instead of the router-level list only. The sync vs async parity cell now asserts the backup's text on both twins.
* test(router): annotate the new fallback test locals as Final and flatten the leak check
* fix(proxy): skip null key and team router settings when merging per-request overrides
* test(proxy): annotate the null router settings test locals and drop a redundant comment
* fix(router): keep include_fallback_errors off the provider call on the sync Router path
Router.completion spread include_fallback_errors into litellm.completion, so OpenAI
answered 400 Unknown parameter and a fallback walk lost the backup's reply. Only
Router._acompletion popped the router-only flags. Both paths now build the provider
call through one shared helper that drops them, and a wire-level respx test covers
each path.
* test(router): pin include_fallback_errors off the wire across the sync, async, and proxy paths
* fix(anthropic): let /v1/messages mid-stream failures reach the proxy failure boundary
AnthropicStreamWrapper.async_anthropic_sse_wrapper swallowed every upstream
exception and emitted its own Anthropic error frame, so a provider drop on a
streamed /v1/messages call never reached the proxy's streaming boundary — no
failure spend row, no failure callbacks.
Re-raise when the stream is proxy-managed (detached failure hook armed) so
async_streaming_data_generator runs post_call_failure_hook once and serializes
the error frame; when consumed standalone (SDK litellm.messages path), run the
logging object's async failure handler and keep the client-facing error frame.
Fixes#44742
* test(anthropic): satisfy PT012 single-statement rule in mid-stream error test
* fix(anthropic): dispatch both sync and async failure callbacks
* fix(anthropic): dispatch both sync and async failure callbacks
* test(anthropic): move the regression cases into the mapped test file
* refactor(anthropic): read the public detached-failure hook and drop the explanatory comments
* test(anthropic): drive the mid-stream failure tests through a real logging object
* fix(anthropic): re-raise the provider error from the chat wrapper envelope so the router keeps its pre-content retry
* test(integration): cover mid-stream failure bookkeeping of bridged /v1/messages streams
Nine integration cells for the chat bridge: raw httpx and the Anthropic SDK (sync and async) streams that fail after the first content delta now land exactly one failure SpendLogs row carrying the provider's error class, the non-streaming 500, the response-cache twin and a client that leaves mid-stream keep their bookkeeping, a 24-stream burst lands every call id once, and the standalone SDK stream reports the provider error to failure callbacks once.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(streaming): stop re-wrapping a bridged stream's MidStreamFallbackError
A chat completion served over the Responses API already gets its mid-stream
error wrapped by the Responses iterator. The chat stream wrapper wrapped it a
second time, and the router unwraps one layer, so the client saw the inner
sentinel (message prefixed litellm.MidStreamFallbackError, type null) instead
of the provider's RateLimitError. The chat wrapper now re-raises an already
wrapped error untouched.
* test(streaming): type the bridged stream regression test's locals
* fix(streaming): rebuild a bridged mid-stream error with the outer wrapper's bookkeeping
* test(integration): cover the bridged stream error typing on every surface
Checked-in audit cells for the Responses bridge: chat completions through the OpenAI SDK and httpx, /v1/messages through httpx and the Anthropic SDK, native /v1/responses, the litellm and Router SDK stream paths, and a chaos file with a mixed burst, a worker SIGKILL and a proxy SIGTERM against an owned two-worker proxy. Every cell scripts the provider through a wire server and asserts the caller's body, the upstream's received requests and the spend row by id.
* test(integration): read bridged chat content through one helper
* test(integration): build the bridged fallback config without mutating the loaded yaml
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(cost): price batch image completion tokens at image batch rate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover default vertex batch output transformation for image cost
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): sync model prices schema
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(cost): simplify batch completion cost with rate helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): keep global batch pricing fallback for image-only deployment rates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(test): rename connect tunnel helper to https redirect
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): merge deployment batch rates over global pricing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(model_prices): put flash-image batch image rate on the GA row
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(cost): drop redundant comment in batch fallback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(cost): use _batch_or_half for the batch image rate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost): price nano banana 2.1 fixtures off the real cost map rows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost): use the vertex 4k image token count in the batch cost test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): never store or serve a chat completion with no choices
A provider response with empty choices was written to the response cache and served on every identical request until the TTL ended, with no provider call in between. The cache now skips storing such a response and treats an already stored one as a miss, so the next request goes back to the provider and its answer replaces the entry.
* fix(caching): skip responses with no output on the Responses API and Anthropic Messages too
* fix(caching): skip streams and stored entries that carry no output
A chat or text completion stream whose chunks carried no choice is closed
by the stream wrapper with one empty choice of its own, so the assembled
response passed the choices check and was cached. The assembled stream is
now judged on its content: a stream with no text, tool call, or other
output in any choice is never stored, on the async and sync writers alike.
The Responses API stream writer and the Anthropic Messages stream writer
apply the same no-output check before storing.
A stored entry with no output read through the worker memory tier is now
evicted from that tier on the miss, so the next read reaches Redis where
the refill lands; the text completion and messages writers only write to
Redis, and the memory copy otherwise kept missing until its own TTL.
* test(integration): response cache cells for answers without output
Deterministic cells for the response cache on every unified endpoint,
streamed and not, through the OpenAI and Anthropic SDKs and raw httpx,
plus the sync SDK paths, stale entries, malformed answers, per-request
TTLs, cache delete, and chaos (Redis stopped or paused mid burst, a
worker killed, in-memory cache mode). The scripted upstream counts only
POSTs as deployment calls, since the proxy's boot-time GET /v1/models
discovery of a config deployment is not one.
* test(caching): pin the stored entry timestamp in the worker-copy test
* test(integration): drop the restating comments from the chaos cells
* test(integration): close the breaker on the first call after the Redis restart
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(chatgpt,github_copilot): refuse device-code login when an event loop is running
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(github_copilot): drop stray whitespace change
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(chatgpt): bound token refresh timeout and drop placeholder assignment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(chatgpt,github_copilot): keep the token file path out of the event-loop 401 message
* fix(auth): refuse device-code login from worker threads too
/v1/messages runs its handler in an executor thread, where the
running-loop check never fires, so a chatgpt or github_copilot model
still started the interactive device-code login there and the request
hung for up to 15 minutes. The guard now also requires the main thread,
so the login only runs where a human can actually answer it.
* test(chatgpt): keep authenticator tests out of the real token directory
* fix(chatgpt): keep the 5 second connect timeout and the operator's request_timeout on the token refresh call
* test(integration): cover the device-code login guard on the proxy and the SDK
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(caching): count tool_call cache_control marks in the injection census
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): remove cache census casts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): only skip injection on message or content marks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): skip injection on messages whose tool calls carry marks
Reverts 9f08d8aef8. A default 5m mark injected on assistant text lands
before the client's 1h tool_use mark, which Anthropic rejects with a 400
because a 1h breakpoint must not follow a 5m one. Keeping the full census
in the skip check leaves the client's tool_call breakpoint as the only
one on that message.
* fix(caching): count every client tool_call cache mark in the breakpoint census
The census gated tool_call marks on type function and dict shape, so a client mark on a call without a type or with a string cache_control slipped past the count and injection overflowed the 4 breakpoint cap. Count any non-None tool_call mark, keep the server tool exclusion, and add integration cells for the capped surfaces, the yaml stand-down, Bedrock and Gemini, and router affinity
* test(integration): hold the upstream so the worker kill lands mid-burst
* refactor(caching): reuse the transform's server tool lookup in the breakpoint census
The census now calls the same helper the Anthropic transform uses to decide
whether a marked tool call becomes a server tool block, so the two cannot
drift apart. The owned-proxy burst test waits up to 90 seconds for the burst
to reach the wire before it kills a worker
* refactor(anthropic): move the server tool rebuild check under llms/anthropic
* test(integration): audit the tool call mark census across chat, messages, responses, and chaos
Adds the /audit cells for the breakpoint census on assistant tool_calls marks: the
Responses stream bridge, the OpenAI and Anthropic SDK clients, in-process Pydantic
messages, response cache twins, request-level points, a provider 401 on a capped request,
malformed provider_specific_fields and tool_call ids, null or empty points, a points
update mid-burst, a proxy restart mid-burst, and a worker SIGKILL that picks the worker
holding the burst's upstream connections
* test(integration): close the SDK clients the cache census cells open
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(vertex_ai): forward system and tools to partner model count_tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): avoid mutable token request construction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): return partner count_tokens provider errors as values so the proxy falls back locally
* test(integration): cover Vertex AI partner count_tokens forwarding and local fallbacks
Add wire-level cells for /v1/messages/count_tokens, /utils/token_counter,
/v1/responses/input_tokens and the Gemini countTokens route on a Vertex AI
Claude deployment: the system prompt and tools reach the partner
count-tokens endpoint verbatim, null fields stay out of the body, malformed
tools are rejected before any peer call, peer, token-endpoint and connection
failures fall back to the local tokenizer unless disable_token_counter is
set, generation on the same deployment keeps working, and concurrent bursts
survive a peer outage, a slow peer and a worker SIGKILL. The sdk cells cover
litellm.acount_tokens the same way.
The _support/process.py and _support/client.py harness files are brought to
main's content so the self-booting cells read INTEGRATION_PROXY_READY_SECONDS
instead of a fixed 70 s boot budget.
---------
Co-authored-by: jesus <jesus@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): stop the proxy_server app fixture leaking LITELLM_LOG
The session app fixture set LITELLM_LOG=ERROR with os.environ.setdefault and never removed it, so later tests on the same xdist worker inherited it. test_drop_params_env_var spawns a subprocess with os.environ and lost the warning it asserts on. Scope the variable to the import with a MonkeyPatch context
* test(secret-detection): give the hand-built redaction request an ASGI path
Since #43975 _read_request_body checks the route path via request.scope, and a scope without path raised KeyError that was swallowed into an empty body, so chat_completion failed with a missing messages parameter. Real ASGI scopes always carry path
* test(integration): isolate litellm callback lists per sdk test
usage-based-routing-v2 Routers register their selector in litellm.callbacks and nothing removes it, not even Router.reset(). The counter TTL and Redis service metrics tests left their selectors behind, and the next usage routing test ran their pre-call checks against its own rpm=1 deployments, raising "Deployment over defined rpm limit". An autouse fixture now gives each sdk test copies of the callback lists and restores the originals afterwards
* test(integration): keep the owner-lookup fault proxy off the shared read replica
The owned proxy points DATABASE_URL at a scratch database but inherited
DATABASE_URL_READ_REPLICA from the replica job, so auth read the shared
database and rejected the freshly created key with token_not_found_in_db.
Drop the replica variable like the other scratch-database owned proxies
* test(integration): request every seeded key in the team owner breakdown
The aggregated team activity endpoint now caps breakdown.api_keys at the top
100 keys by default (#43398), so the 300 seeded keys came back as 100 rows.
The test guarantees each key is reported with its own owner, so ask for an
api_key_limit that covers all seeded keys
* test(integration): give every owned Redis its own port in the redis-cache container
On CircleCI every owned Redis ran on the fixed port 16379 inside the shared
redis-cache container. When an earlier server still held that port, the new
one failed to bind, readiness pinged the old server, the pidfile read failed
and cleanup then reported "Owned Redis still serves after shutdown"
Reserve an ephemeral port for the docker-exec path the same way the local
binary path already does, and refuse to start when something already serves
the chosen port so the failure names the real cause
* test(e2e): skip the Vertex Mistral partner case the e2e project cannot reach
The e2e Vertex project gets a 404 publisher model not found for vertex_ai/mistral-small-2503, so the case can only fail
* test(e2e): skip the Vertex gpt-oss partner case the e2e project never serves
vertex_ai/openai/gpt-oss-120b-maas has hit a 60s read timeout with no response headers on every run in the e2e Vertex project since the case was ported, and no other Vertex partner chat model passes there to switch to
* test(e2e): check only stored message content for a leaked card number
The Presidio spend-log check ran the card-number pattern over the whole serialized response, so a Luhn-valid usage.cost float (0.0003466000000000001) failed the streaming /v1/messages case although the stored content was <CREDIT_CARD>. The check now reads the content and text strings of the stored response, which is where a raw card would land, and still requires the placeholder there
* test(e2e): assert the proxy decodes token-array embeddings for titan
The port in #44120 carried over a legacy SDK-direct test that expected Bedrock to reject token ids with a 400. Through the proxy, /embeddings decodes token arrays to text for providers that cannot embed tokens, so titan answers 200. The test now sends a token array and its decoded sentence and requires the two vectors to match, which fails if the proxy stops decoding or decodes with the wrong tokenizer
* test(e2e): run the Bedrock extended-thinking round trip on a model that honors enabled thinking
us.anthropic.claude-sonnet-5-5 is adaptive-only, so litellm sends thinking.type=enabled with a 1024 budget as adaptive with low effort, and Bedrock returned no reasoning blocks on 5 of 5 identical Converse calls (boto3 direct agreed). us.anthropic.claude-sonnet-4-6 accepts the legacy shape verbatim and returned reasoning on 5 of 5. The non-thinking Bedrock case stays on sonnet-5-5
* test(proxy): stop unit modules forcing DEBUG logging into the event-loop lag tests
Five tests/unit modules set verbose_proxy_logger to DEBUG at import, so every xdist worker that collected them logged the 2.4MB pass-through response from a worker thread, and secret redaction of that line held the GIL for ~0.8s+ inside the timed window. The lag tests now pin the LiteLLM loggers to WARNING and freeze gc while timing, and the module-level DEBUG overrides are removed
* test(e2e): cite the tokenizer and date behind the titan token-array fixture
* test(e2e): let migration seed replicas finish their request-log indexes before cloning
Since #43948 a serving proxy builds the two LiteLLM_SpendLogs indexes on a background thread after it reports ready. The seed fixtures stopped the replica at readiness, so every cloned legacy database lacked an index no real deployment would be missing, and the v2 baseline diff refused it. Seeds now wait until both indexes exist and are valid in the database's schema
* test(passthrough): give the pass-through MockRequest an httpx URL and ASGI scope
#43626 made get_request_route read request.scope during pass-through kwarg setup; the MockRequest in tests/unit/passthrough had neither a scope nor a URL object, so both stream-param tests raised before reaching the code they check. Mirrors the repair #43626 made to the tests/pass_through_unit_tests fake
* test(integration): ignore foreign allow_all_keys MCP servers in the access matrix tool list
test_toolset_gateway_url_serves_a_team_granted_toolset_to_a_key_without_its_own_grant (#43908) registers an allow_all_keys server on the shared gateway, and allow_all_keys servers are listed to every key by design, so a matrix case running on another xdist worker at the same time saw its tools. The matrix now drops tools of allow_all_keys servers it did not create, read from LiteLLM_MCPServerTable before and after listing, and still compares everything else exactly
* test(integration): move legacy proxy, router and Redis tests into tests/integration
Port 39 legacy tests to the integration tier that owns them, running against
the scripted upstream, local Postgres and Redis, test-owned wire peers and
owned proxies. Delete 5 legacy tests whose contract is already owned by an
existing integration test, and remove the legacy functions, files and helpers
left unused.
* test(integration): cover recovery of a spent key after its budget is raised
* ci(circleci): test Redis behavior against local Redis and print short tracebacks
The redis_caching_unit_tests job ran three legacy files against the shared remote Redis. The DualCache and
batch-read logic that never needed a server now lives in tests/unit/caching/test_dual_cache.py with a mocked
RedisCache, and the behavior that does need one (the increment-with-floor Lua script, read-through, deletes,
batch reads) moved to tests/integration, which starts a local Redis. test_returned_settings only read
REDIS_PORT and is replaced by a unit test of Router.get_settings
CircleCI pytest runs now use --tb=short so failure output stays readable in the test results tab
* ci(integration): print short tracebacks from run.py and allow Redis in the sdk shard
* refactor: clean up fresh tech debt from 2026-09-29
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(routing): pin usage-based routing Redis reads through the proxy and SDK
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): optional Anthropic tool properties stay optional on the OpenAI Responses wire (Pylon #6619)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Bedrock InvokeModel count-suffixed cache usage fields are reported and charged (Pylon #6708)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): anthropic messages honors the deployment request timeout (Pylon #6505)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop client_metadata before the Bedrock Converse body reaches the provider (Pylon #6645)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): repeat Bedrock requests under one session name assume the role once (Pylon #6681)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): messages stream keeps include_usage off the Responses wire with always_include_stream_usage (Pylon #6466)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): clamp sub-16 max_tokens to the Responses API floor instead of 400 (Pylon #6539)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): forwarded client x- headers reach the provider on /v1/responses (Pylon #6565)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Anthropic messages stop_sequences reach OpenAI-compatible providers as stop (Pylon #6536)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Codex namespace tools reach a chat upstream flattened and round-trip through /v1/responses (Pylon #6409)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): keep Claude 4.6 legacy thinking budget_tokens on /v1/messages (Pylon #6727)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): nvidia nim ranking keeps image passages and applies top_n without sending top_k (Pylon #6401)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): tpm-only model rejects priority traffic once recorded tokens reach the model tpm (Pylon #6344)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): reasoning-only chunks open an Anthropic thinking block at index zero on /v1/messages streams (Pylon #6337)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): file content streams to the client before the upstream finishes sending (Pylon #6315)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): agent whose card lives only at agentCard/v1.0 is reached with bearer auth (Pylon #6249)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): vertex batch create returns a batch when outputInfo is null (Pylon #6374)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): fireworks session id is sent as x-session-affinity and cached tokens land in spend log metadata (Pylon #6220)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): advisor sub-call failure does not cool down the executor deployment (Pylon #6212)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Gemini /v1/messages cache_control creates cachedContent with Anthropic ttl (Pylon #6221)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock_mantle max_output_tokens below 16 is clamped before reaching Mantle (Pylon #6262)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): missing thinking signature 400 on /v1/messages retries without thinking blocks (Pylon #6222)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): format Gemini messages cache_control wire test (Pylon #6221)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): rebuilt shared aiohttp session keeps the configured keepalive timeout (Pylon #6387)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): sagemaker_chat signs the inference component header and sends hf_model_name as the body model (Pylon #6187)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Bedrock Converse DeepSeek drops Anthropic thinking and sends V3 reasoning_effort raw (Pylon #6149)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): concurrent team model TPM requests are reserved before the provider call (Pylon #6075)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): /v1/messages honors the configured timeout against a stalled upstream (Pylon #6025)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Codex additional_tools input items reach Bedrock Mantle as top-level tools (Pylon #6012)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): advisor api_base without api_key never sends the proxy Anthropic key to the caller host (Pylon #6226)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): rerank responses carry call id, latency and cost headers (Pylon #5981)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): parse the outbound Anthropic body with the typed JSON adapter (Pylon #6025)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Bedrock Knowledge Base search forwards userContext to the Retrieve body (Pylon #5991)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Marengo 3.0 text embeddings reach Bedrock nested under inputType (Pylon #5949)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): sub-16 max_tokens over a responses deployment reaches OpenAI as 16 (Pylon #6008)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): midturn system correction reaches the OpenAI Responses wire via /v1/messages (Pylon #6449)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): concurrent requests over a key tpm limit are rejected before reaching the provider (Pylon #5737)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): chat to responses bridge keeps deployment AWS credentials for Bedrock Mantle SigV4 (Pylon #5870)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): vertex gemini stream split across many fragments completes without stalling the proxy (Pylon #5838)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock mantle /v1/messages stream keeps stream true and relays SSE events (Pylon #5596)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): large chat payloads are released from worker memory after the request ends (Pylon #5920)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): streaming success logs v3 rate limit remaining values for callbacks (Pylon #5767)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): a database created search tool backs Anthropic web search interception (Pylon #5669)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): register july provider regression contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): make july provider regression tests deterministic under cache and worker sharing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop order-fragile worker memory probe pending a real retention regression check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): apply ruff import sorting and formatting
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop stale contract entry and pass question to advisor executor
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): use tiktoken-backed executor model in advisor tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The check spins up a hypercorn TLS peer and drives the SDK's own httpx
handlers at it, so it needs litellm importable, hypercorn installed and a
loopback socket. It lived under tests/e2e, whose Buildkite runner image
installs neither litellm nor hypercorn by design (the suite drives a
remote proxy over HTTP), so every scheduled e2e build since #230 failed
to import the module and pytest reported it as a collection error. The
unit tree bans sockets, so it does not belong there either
tests/integration is the CircleCI tier built for real TCP against local
protocol peers. This adds an sdk shard to it for cases that exercise the
SDK's clients with no gateway in the path, registers the two HTTP/2
nodes in the contracts manifest, and adds the shard to the CircleCI
matrix. The test now flips the feature through LITELLM_HTTP2 (the user
surface) instead of patching module attributes, and asserts the version
the peer observed on the wire next to the one the client reports
