Commit graph

6 commits

Author SHA1 Message Date
Yujong Lee
dc097107b2 feat(db): add the SQLx pool foundation with the writer pool
Some checks failed
Cost map pins recheck / rerun-cost-map-guard (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust DB / db-queries (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
NativeDatabase.connect opens a lazy SQLx writer pool from an Endpoint that
Python resolves from the final DATABASE_URL, read the way Prisma reads it:
Unix socket host, sslmode / sslaccept / sslcert with Prisma's verification
rules (prefer + strict asks the server once whether it offers TLS),
sslidentity / sslpassword handed over as PEM, schema, options, pgbouncer mode,
pool_timeout (0 waits forever) and socket_timeout. Options SQLx cannot send
are logged instead of failing boot

ConnectionBudget splits each worker's connection_limit between Prisma and the
native pool, so the budget an operator sized never grows. The Rust share is the
hard-coded RUST_RESERVED_CONNECTIONS and no setting names Rust, SQLx or Prisma.
It is 0 until a ported query needs the pool, so Prisma's URL is exactly what it
was, no native pool opens and nothing connects through SQLx. A connection_limit
that is not a positive integer is never split and goes to Prisma as before.
Once the share is above 0 Prisma keeps at least one connection, and pools are
tagged litellm-prisma and litellm-native in pg_stat_activity. Python keeps
minting RDS IAM and Entra tokens and hands each one to the pool

statement_cache_size sets the native statement cache (Prisma's default 100) and
pgbouncer=true turns it off. sqlx-postgres 0.9.0 still names every statement when
the cache is off, so Statements runs queries unnamed in that case, like Prisma,
until a release carries transact-rs/sqlx#4337. An application_name the operator
set is used on both pools

The SQL query files are pinned to LF so Windows checkouts hash them like .sqlx,
and the crate lists .sqlx in its package include so the core sdist builds offline

crates/db/AGENTS.md tracks parity with the Python Prisma stack

The bridge now links litellm-db, so the editable install compiles its queries.
The Rust DB workflow builds that install from the checked-in .sqlx metadata,
since it runs before the migrations that the online check needs
2026-11-13 16:50:02 +09:00
yuneng-jiang
1677c2250e
ci: stop PR runs from thrashing the Actions cache and stop smoke compiling from source (#51343)
Some checks failed
unit / guardrails-hooks (push) Blocked by required conditions
unit / jwt-and-keys (push) Blocked by required conditions
unit / key-generation (push) Blocked by required conditions
unit / logging-misc (push) Blocked by required conditions
unit / mcp-oauth (push) Blocked by required conditions
unit / proxy-runtime (push) Blocked by required conditions
unit / proxy-server-core (push) Blocked by required conditions
unit / root (push) Blocked by required conditions
unit / proxy-utils (push) Blocked by required conditions
unit / rust-bridge (push) Waiting to run
unit / ui-unit (push) Waiting to run
unit / docs (push) Waiting to run
unit / helm (push) Waiting to run
unit / OpenAI 2.20.0 compatibility (push) Blocked by required conditions
unit / OpenAI 3.0.0 compatibility (push) Blocked by required conditions
unit / OpenAI 3.25.0 compatibility (push) Blocked by required conditions
unit / redis-py 5.3.1 compatibility (push) Blocked by required conditions
unit / redis-py 6.4.0 compatibility (push) Blocked by required conditions
unit / redis-py 7.4.1 compatibility (push) Blocked by required conditions
unit / redis-py 8.0.1 compatibility (push) Blocked by required conditions
unit / coverage (push) Blocked by required conditions
unit / unit passed (push) Blocked by required conditions
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Issue label sync / sync-issue-labels-tests (push) Has been cancelled
Issue label sync / sync-issue-labels (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
VS Code Extension / vscode-extension (push) Has been cancelled
* ci: stop PR runs from writing Actions caches and remove the sccache GitHub cache backend

PR runs were saving their own cache copies and sccache was making one cache
request per compiled file, about 700 to 2000 per Rust build. Together they
pushed the repository past GitHub's cache storage and request limits, so
restores failed with 429 or found nothing and jobs rebuilt from scratch. The
rust-bridge job that gates every unit shard went from 0% cache misses on
Oct 21 to about 65% on Nov 11.

Caches are now saved only from main and restored everywhere else. The Rust DB
cache gets the same save-if guard the other Rust jobs already have. The lint
python and ui-api-types jobs restore the uv cache that the unit workflow saves
on main, because lint only runs on pull requests and could never save a cache
that another pull request can read. The Terraform provider jobs turn off
setup-go's built-in cache, which saves from every ref, and use a new
cache-go-build action that follows the existing restore-everywhere and
save-on-main pattern.

sccache is removed from the unit, Rust and Rust DB workflows. The Cargo target
directory caches they already restore stay in place.

The smoke diagnostics upload no longer fails a job whose checks passed.

* ci(smoke): install the prebuilt Rust bridge and cache wheels built from source

Every smoke job compiled the Rust bridge inside uv sync, about 4.5 minutes each
in 8 jobs per run, and the Python 3.14 and 3.15 jobs also compiled grpcio-tools,
numpy and jiter from source on every run because no wheels exist for those
versions yet. Smoke only ran on pull requests, so nothing it built could be
saved for the next run.

The Linux smoke jobs now restore the bridge that the unit workflow already
builds once and saves on main. The bridge is abi3, so the same build serves
every Python version. When the restore misses, which happens when a pull
request changes a Rust build input, the job restores the Cargo build cache and
compiles as before. The cache key and restore step move into a
restore-rust-bridge action that the unit workflow's producer job also uses, so
the key is defined once.

Smoke now also runs on pushes to main and prunes the uv cache down to wheels
built from source before setup-uv saves it. Pull requests restore that cache,
so 3.14 and 3.15 stop recompiling the same packages.

The windows-sdk job drops its Cargo cache step. Nothing ever saved a Windows
copy, and saving one from main would add about 2 GB to a cache that is already
near its limit.
2026-11-11 23:52:32 -08:00
Yassin Kortam
adc4c01aa9
chore: move scripts and runtime sources under litellm (#51006)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-11-12 04:19:57 +00:00
Yassin Kortam
db6dfb6bb6
chore: move workspace packages under litellm (#51000)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-11-12 04:19:57 +00:00
Nate Armstrong
6ad4f9cfdf
ci: cache Rust compilation with sccache (#49117)
This should make compilation much faster
2026-10-30 12:14:17 -07:00
yujonglee
554b011787
feat(rust): add typed database query foundation (#48214)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-26 17:05:04 -07:00