Migrate 18 provider test files from tests/test_litellm/llms to tests/unit/llms.
194 kept tests move as-is after mutation testing; 1 test deleted
(test_completion_datarobot_with_environment_variables, env-gated no-assert);
the fixture-only fal_ai cost calculator file is removed.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Postgres-backed smoke job previously exercised one resolver. Running
the default and the legacy variants in it covers v2 now that it is the
default, without losing v1's coverage.
The migrations Job entrypoint (migrations/run.py) has defaulted to v2 with
USE_V2_MIGRATION_RESOLVER=false as the opt-out, and the Helm chart documents
that knob. Proxy startup still defaulted to v1, so the two paths disagreed
about which resolver a deployment runs.
Proxy startup now resolves the same way: v2 unless USE_V2_MIGRATION_RESOLVER
is false or --use_legacy_migration_resolver is passed.
- --use_v2_migration_resolver stays accepted as a no-op that warns, so
existing commands and Helm values do not fail on an unknown option.
- The dedicated Postgres smoke-test job is repointed at the legacy resolver
so v1 keeps real-DB proxy-boot coverage, and the two jobs that deselected
it by name are updated to match the rename.
#39178 reverted an earlier flip because two replicas sharing a database
deadlocked (40P01 / P3018) with neither answering /health/liveliness. That
contention is what #40932 coordinates, which is why this builds on it.
The changed-test gate booted its stage-mirror stack without files_settings
or finetune_settings, so every raw upload with a custom_llm_provider hit a
500, and it exported the whole provider env into the gateways, so the
AWS_ROLE_NAME the assume-role test needs made the GovCloud deployment run
an AssumeRole with its static keys. The gate also deleted its pytest output,
so a red run left nothing to read. The mirror config now carries the
openai, azure, and vertex_ai file settings, gateways start without
AWS_ROLE_NAME, and the workflow uploads the pass logs and junit files with
every secret value, every field of a JSON-valued secret, and their
XML-escaped forms replaced before the raw files are removed.
Route openai/gpt-image-2.5/{flare,sunburst}/text-to-image through the existing GPT Image config with the xhigh and max quality tiers, add a dedicated fal-ai/flux/dev config, and add a Fal image-edit config so /v1/images/edits works for the gpt-image-2.5 and gpt-image-2 edit endpoints. Add flat and quality-by-size keyed pricing rows so spend is non-zero
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Offload the per-block hashing through offload_token_count on both the pre-call
read and the success-event write, hash raw bytes as base64 instead of raising,
drop the unused serialize_object helper, and bind the chained digest, the
message envelope, and the bytes path in the regression tests
The prompt_caching pre-call check keyed a deployment pin on a hash of the
whole cacheable prefix, cache_control markers included. Agent clients
such as Claude Code move the marker to the newest user turn on every
request, so the key changed every turn, the pin never matched, and a
multi-turn session drifted across deployments and lost its provider
cache.
Hash the prefix per content block with the markers stripped, chained so
every block position has a key, and write the pin at the breakpoint
block. Lookup walks back over the last PROMPT_CACHE_LOOKBACK_POSITIONS
positions (a run of tool_use or tool_result blocks counting as one), the
same window the provider probes for a cached prefix, in one batch cache
read. Both sides hash the prefix after base64 truncation so a request
carrying raw image bytes derives the keys the success event stored.
A tools/call on a cold worker listed the target server once and then never
again, so a later caller whose credentials expose a wider upstream catalog
got 404 for tools the first caller never had. Gate the pre-call listing on
whether this worker already exposes the requested tool, so callers with
different catalogs no longer mask each other. Removing the per-server guard
also drops the empty-listing case that re-listed on every call.
A tools entry of only {"type": "function"} has nothing for the Responses
bridge to convert, and the bridge raised a 500 for it where the chat
route returns the provider's own 400. The gate now counts a tool as a
function tool only when it carries a function body or a top-level name,
on every provider the gate serves
The startup tool-name fill skips servers whose upstream wants the caller's
own token (true_passthrough, OAuth discovery), and mcp 2 no longer runs the
list handler before an uncached tools/call, so every uvicorn worker that had
not served tools/list answered 404 "Tool not found" for prefixed tools/call
and the REST server_id route on those servers.
On a resolution miss, execute_mcp_tool now lists the prefix-matched (or
server_id-requested) server once, with the caller's credentials, through the
existing tools/list path, then resolves as before. Listing failures fall
through to the existing 404, a worker that already listed the server never
re-lists it, and a server outside the caller's allowed set is never listed.
