Commit graph

5 commits

Author SHA1 Message Date
Mateo Wang
610443519a
fix(proxy): unset DB environment variables removed from the row on every worker by row revision (#46709)
* fix(proxy): unset DB environment variables removed from the row on every worker by row revision

* fix(proxy): order the environment variables revision migration after main's and set its lock timeout

* fix(proxy): snapshot the environment at the first DB sync even when no environment_variables row exists yet

* fix(proxy): close gaps in DB environment variable reconciliation

Grant the revision sequence to every role so a writer role with only
table grants can still save environment_variables. Unset SSO keys that
only the environment_variables row stored on the next SSO sync, while
keeping keys the SSO settings table holds. Skip a revision-less row once
a revision has been applied, so a stale cached row cannot bring a
removed key back. Keep plaintext and undecryptable values when rotating
the master key instead of dropping them from the row.

* fix(proxy): keep env var revisions monotonic across DB copies and restores

Stamp at least the old revision plus one on update so a copied row keeps
rising, re-read a stale cached row from the writer and apply it when the
database itself went back, unset dropped SSO keys when sso_settings are not
loaded from the DB, and keep env values master key rotation cannot decrypt
exactly as stored

* fix(proxy): skip an env var row read before an overlapping sync that already applied a newer one

* test(proxy): cover a read replica back on an older row and an SSO client removed from DB environment_variables

* fix(proxy): keep config reads up when the writer is down, find the revision sequence by table schema, read the writer before rewriting env vars, and add the trigger under db push

* test(proxy): model the writer database in the retry policy config update fake

* fix(proxy): recheck the environment_variables row revision right before applying it

A sync that started later could read a stale environment_variables row
before an earlier sync applied a newer one, then pass the read-order check
and apply the stale row, bringing back keys the newer row removed. The
behind-the-applied-revision check now runs at apply time, re-reading the
writer when the row is behind, and a writer row read while another sync
applied a newer revision is dropped.

The config sync tests now inject a fake database client and use the real
get_config_param and its cache instead of replacing get_config_param.

* test(proxy): keep encryption real in the SSO replica-lag test

* test(proxy): inject a fake Prisma client in the env var replica-lag test

* fix(proxy): keep the applied environment variables when the writer database is out of connections

* test(proxy): cover a writer database out of connections in the env var integration tests
2026-10-17 03:51:05 +00:00
Yassin Kortam
685c3ec6a2
fix(proxy): finish prisma engine requests when their caller is cancelled (#46503)
Cancelling a task mid-request closed the query engine socket, the engine
handed the still-busy pool connection back, the next transaction start
queued its BEGIN behind the orphaned statement, and when that waiter was
cancelled too the engine panicked on its dropped reply channel
(interactive_transactions/actors.rs) and exited under the shutdown spend
flush, losing daily user and team spend for requests that had returned 200.

Every engine request now runs in its own task behind asyncio.shield so only
the caller is cancelled, and a transaction that opens after its caller left
is rolled back.

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 15:09:16 -07:00
devin-ai-integration[bot]
e6639d15b5
fix(proxy): always exit when database setup fails at boot (#44141)
Remove the ENFORCE_PRISMA_MIGRATION_CHECK opt-in. When PrismaManager.setup_database returns False (database unreachable, connection retries exhausted, or prisma migrate deploy failing after retries) the proxy now always prints the red failure message and exits 1 instead of serving requests against a database whose schema may be behind the code. The --enforce_prisma_migration_check flag stays as a hidden no-op that prints a one-line deprecation warning so existing container args keep parsing; the env var is no longer read anywhere. The standalone migration entrypoint always runs run_server(("--skip_server_startup",)), and the integration launchers drop the flag.

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-01 21:51:06 -07:00
devin-ai-integration[bot]
c66c8288c3
fix(proxy-extras): build the SpendLogs indexes in the migration job instead of in migrations (#43948)
The two SpendLogs index migrations shipped in v1.103.0 each break one table shape: the plain CREATE INDEX holds a SHARE lock on a large unpartitioned table and the CONCURRENTLY one fails with 0A000 on a partitioned parent. Both files are now inert and the indexes are built by a table-driven, shape-aware step after migrate deploy: CONCURRENTLY on a plain table, ON ONLY the parent plus per-partition CONCURRENTLY and ATTACH PARTITION on a partitioned one. The migration job builds them synchronously and exits non-zero on failure; a serving proxy that ran migrate deploy itself builds them in the background off the readiness path. A valid index of the same definition under another name is renamed and reused, an invalid one is rebuilt, and extra copies are reported with their DROP INDEX statement instead of being dropped. The migration checker rejects any CREATE INDEX on LiteLLM_SpendLogs or LiteLLM_ErrorLogs in future migrations

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-01 14:12:22 -07:00
devin-ai-integration[bot]
24584d3d3d
test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)
* test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): keep tuple identity in proxy state restore and fix misc target paths

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-01 18:14:24 +00:00
Renamed from tests/test_litellm/proxy/db/test_prisma_client.py (Browse further)