Mirror the Redis vertical slice for S3Cache: a litellm-cache-s3 crate built on aws-sdk-s3 with path-style custom endpoints, python-identical put_object metadata (cache-control, expires, content headers), expires-aware get_object, and no-op flush/unsupported test_connection. Wire it through python-bridge config projection, facade guards, test handle, and binding dispatch, plus an in-process S3 stub and parity tests.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
/config/field/info and /config/list already report per-key source on main,
so this drops the branch's versions of those and keeps /alerting/settings,
/get/ui_settings and /router/settings.
Read endpoints no longer write the freshly read database row back into the
shared settings store; the reload path already keeps it current, and a GET
that mutates global state leaks across callers.
Regenerates the lazy OpenAPI snapshot on Python 3.12, matching CI, and the
dashboard API types for the two new response fields.
Three holes in the new suite, all of which let a test pass without proving
what its name claims:
- A migration recorded twice, once per replica, each with
applied_steps_count = 1, slipped past both the step-count check and
migration_names(), which collapses the history into a set. Reject
duplicate migration_name rows outright.
- auth_traffic only asserted the failures it had seen by the time
keep_serving hit its target. A request failing after that, or on the
other replica while the test waited on one stream, was recorded and
never read. Assert the recorded failures once the thread has joined.
- The rolling test warmed the baseline replica's virtual-key cache before
the upgrade, and that cache holds for 60 seconds by default
(UserAPIKeyCacheTTLEnum.in_memory_cache_ttl). The candidate migrates
well inside that window, so the post-upgrade requests could be served
from cache without ever repeating the whole-row token lookup that the
stale prepared statement breaks. Drive the baseline replica with a key
minted after the schema moved, which it has never seen and must resolve
from the database.
Re-ran against v1.101.0 -> v1.102.0: 6 passed.
* fix(bedrock): send s3BucketOwner on batch input and output data config
Resolve s3_bucket_owner from litellm_params, then optional_params, then
AWS_S3_BUCKET_OWNER and emit it on both S3 data configs so cross-account
batch buckets pass Bedrock ownership validation. Omitted when unset
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(bedrock): build batch output config with explicit returns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The install smoke test starts the proxy on test_config_no_auth.yaml, which has no master key on purpose, and #42019's boot check now refuses that, so the three installing_litellm_on_python jobs have been red on main since 2026-09-20. Pass the documented local-dev override to the proxy child so the test keeps its no-auth config and the boot check stays as it is
* feat(policy_engine): add default fallback policy attachments
A policy attachment with default: true applies only when no non-default
attachment matches the request, so an opt-in guardrail policy replaces the
fallback one instead of running alongside it. Supported in config.yaml,
/policies/attachments, the Admin UI Attachments tab and the resolver
(matched_via is prefixed with default:).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(policy_engine): ignore inapplicable non-default attachments when selecting defaults
A non-default attachment whose policy is missing or whose condition does not match the request
no longer suppresses default attachments. The impact preview marks default counts as an upper bound
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(policy_engine): accept any sequence of policy names in condition matching
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(policy_engine): resolve policies once and apply fallback semantics in get_matching_policies
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The migration e2e harness only ever used one image: it seeded the database
with the candidate build and then applied synthetic migrations on top. That
proves the migration machinery (locking, crash recovery, legacy baselining,
pooling) but never executes the real schema of release N against the real
migrations of release N+1, which is the path operators actually run.
Adds a baseline image alongside the candidate, so a test can seed with a
published release and upgrade with the build under test.
Suites:
- test_upgrade.py: the candidate applies the pending release migrations,
keys minted by the baseline release survive, and concurrent replicas
upgrade a baseline database exactly once.
- test_rolling_upgrade.py: a baseline replica keeps serving virtual-key
auth while the candidate migrates underneath it, and both releases serve
and resolve each other's keys during the overlap. This is the reported
failure: a new column on LiteLLM_VerificationToken invalidates prepared
plans on pods still running the old release, which the proxy reads
whole-row, and auth starts failing until those pods leave service.
- test_shaped_database.py: the upgrade completes and preserves rows on a
populated spend log, rather than on the empty database every other
migration test starts from.
Every upgrade assertion is gated on the candidate having actually applied
migrations the baseline had not, so a stale pin fails loudly instead of
passing on an empty delta.
CI adds two jobs to the migration_startup workflow. The baseline defaults
to a committed release pin and is overridable per pipeline, matching how
migration_candidate_image already works; only the upgrade jobs pull it.
Verified against a real v1.101.0 -> v1.102.0 upgrade: 6 passed, with the
baseline seeding 165 migrations and the candidate applying the 6 that
landed between the two releases.
