Commit graph

8 commits

Author SHA1 Message Date
yujonglee
677205b3f5
fix(proxy): share ownership permissions for spend logs and traces (#44239)
* refactor(proxy): extract shared spend log read policy

* test(proxy): use named bindings for spend scope regression

* test(proxy): reuse existing spend log query harness

* test(proxy): cover spend log permission lookup adoption

* chore(proxy): relocate existing spend query baseline

* refactor(proxy): make scope query returns explicit

* refactor(proxy): inject deferred log permission lookup

* test(proxy): cover teamless management compatibility lookup

* refactor(proxy): compose user and team log grants

* refactor(proxy): share generic authorization composition

* refactor(proxy): compose trace read permissions

* refactor(proxy): centralize spend and trace authorization

* refactor(proxy): strengthen spend and trace scope types

* refactor(proxy): flatten log read scope into owned logs

Replace the AnyOf grant tree with a flat OwnedLogs(user_id, team_ids) scope,
and OwnedTraces(logs, api_key_hash) for traces, since every consumer flattened
the tree back into that shape.

A caller with no user id now gets an empty scope instead of matching ownerless
rows through Prisma's IS NULL. The dead request_id guard in ui_view_spend_logs
is removed, and the management facets inject the log team lookup and reuse
read_scope_sql instead of the list shim.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(proxy): run spend scope tests through one SQLite emulator

Replace the string-matching payload emulator and the hand-rolled Prisma where
interpreter with one SQLite helper that runs the real scope SQL. Session scope
tests now go through the endpoint, including the no-user caller that must not
match ownerless rows. Drop duplicated lookup-failure and trace mapping cases.

load_permitted_log_team_ids returns no teams without a database instead of
relying on the resolver's broad except.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(proxy): unify log and trace ownership permissions

* test(tracing): align fixtures with ownership read scopes

* refactor(tracing): align query scopes with row ownership

* refactor(spend): make ownership SQL predicates explicit

* test(spend): validate ownership SQL against PostgreSQL

* docs(traces): drop key-row visibility from query help guide

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(spend): reach the empty-memberships branch in team lookup test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(ui): regenerate dashboard API types

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-03 01:38:54 +00:00
yuneng-jiang
b877a38e5f
fix(proxy): stop queued registry read-throughs spending the resync budget (#44277)
* fix(proxy): stop queued registry read-throughs spending the resync budget

RegistryReadThrough.attempt serializes misses behind one lock, but every
request that was queued behind the first one still spent a unit of the
20-per-5s resync budget and re-ran the DB resync, even though the first
request had already loaded the object. A burst of more than 20 requests
for a model created on another worker therefore exhausted the budget and
the rest got 400 "Invalid model name".

attempt now checks whether the key is already loaded once it holds the
lock and returns early without touching the budget. Models check the
router's model names and deployment ids, guardrails and agents reuse
their existing registry lookups.

* test(proxy): gate the queued read-through test on events and cover each registry's loaded check

The queued-requests test now holds the first resync on an asyncio.Event instead of
a timed sleep and records calls in a recorder with tuple and frozenset state. New
tests show the model, guardrail and agent read-throughs each answer an object that
is already loaded without reading the database, so rewiring any registry's loaded
check now fails a test

* test(proxy): keep the queued read-through recorder inside its test and type the agent registry fixture
2026-10-02 18:12:56 -07:00
yujonglee
ec605826d4
feat: improve trace ingestion and trace details (#43975)
* refactor: separate OTLP HTTP decoding from trace codec

* feat: complete trace ingestion and read paths

* fix: encode OTLP protobuf errors in Rust

* fix: raise OTLP body limit to 16 MiB

* test: cover OTLP auth body parsing boundary

* refactor: parse OTLP media type into enum

* fix: enforce OTLP body size at HTTP boundary

* perf: preserve shared OTLP metadata across ingestion

* bench: compare owned and shared trace resource fanout

* refactor: extract shared storage and Python conversion caches

* refactor: keep shared storage owned by traces

* test: keep trace loopback coverage in Rust

* test(proxy): adapt trace coverage to injected access context

* fix(tracing): satisfy stacked branch lint checks

* refactor(tracing): use immutable ingestion payloads

* fix(tracing): declare native error encoder export

* test(proxy): resolve trace access through dependency

* fix(tracing): align merged normalizer types and bridge tests

* fix(tracing): address ingestion and diagnostic review findings

* fix(proxy): preserve body parsing for partial request scopes

* test(proxy): use valid HTTP scopes in request fixtures

* test(proxy): complete auth request flow scopes
2026-10-01 13:45:33 -07:00
devin-ai-integration[bot]
a76b59db9f
test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-01 18:23:31 +00:00
tin-berri
de06c93767
feat(router): opt in to prompt-cache cost routing (#43232)
* feat(router): opt in to prompt-cache cost routing

* fix(router): address prompt-cache routing review

* ci: include cache-routing regressions in coverage
2026-09-26 18:13:13 -07:00
devin-ai-integration[bot]
e0fb89bc82
fix(proxy): keep the submitted body out of 422 validation errors (#43231)
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-25 17:05:53 -07:00
Mateo Wang
25fb7810c2
fix(spend): attribute CLI session spend to the per-user cli-session alias instead of the hashed session token (#40541)
* fix(spend): attribute CLI session spend to the per-user cli-session alias instead of the hashed session token

A CLI session token is a fresh random secret on every login, so since v1.99 each
login's spend rows carried a different sha256 hash as api_key and the usage APIs
could resolve neither key_alias nor user_email for them. Spend rows and logging
callbacks now attribute a session request to its stable alias,
cli-session-<user_id>, and the usage endpoints derive that alias and owner from
the key itself instead of scanning for a matching digest

* fix(spend): resolve the CLI session team from the user's first team in usage metadata

A cli-session key carries no team of its own in the DB, so the usage
breakdown showed team_id None for it and the export grouped it as
Unassigned. The login attaches the user's first team to the session, so
the recovery mirrors that rule for cli-session keys only.

* fix(spend): claim the session team only for a single-team user

The CLI login attaches a team on its own only when the user has exactly
one; a user in several teams picks one per login, so usage metadata for
the alias would otherwise name a team the login may not have used.

* test(pass_through): mark the mocked auth object as a plain key

The logged key follows the alias only for a session token; a bare
MagicMock reads as one, so the test names the field it relies on.

* fix(spend): attribute CLI session pass-through, queue, and managed batch spend to the cli-session alias

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(spend): only treat the exact cli-session-<created_by> value as a batch key alias

A managed object row written by an older build can still carry the raw per-login
session token, which shares the cli-session- prefix. Matching on the prefix alone
would have surfaced that token as a trusted alias and persisted it verbatim in the
batch cost spend log, so the alias check now requires the exact per-user value and
every other prefixed value keeps going through redaction

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(spend): log proxy executed batch rows under the cli-session alias instead of the session token

_row_metadata set user_api_key from the raw bearer token while user_api_key_hash carried the alias, so the spend log redaction rejected the alias as untrusted and hashed the random session token instead

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(spend): attribute semantic search embedding spend to the cli-session alias

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(spend): scope /key/spend/report for a CLI session to the cli-session alias

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(spend): use the cli-session alias for websearch spend, prometheus failure labels and the parallel limiter

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(spend): drop explanatory docstrings on get_logged_api_key and attach_user_details

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(spend): only recover cli-session usage keys whose suffix is a known user

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 18:21:47 -07:00
devin-ai-integration[bot]
248f0eb159
ci: move tests/proxy_unit_tests to tests/unit/proxy and run the proxy-db shards from litellm-tests (#42903)
* ci: fix the litellm-tests unit job with sysmon coverage, an env allowlist and coverage upload on failure

* test: replace key-dependent proxy, enterprise and mcp unit tests with synthetic values and integration and e2e coverage

* test: drop key reads at the legacy proxy, enterprise and mcp paths and wire the gemini pass-through split

* ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests under their legacy flags

* ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests under their legacy flags

* ci: move tests/proxy_unit_tests to tests/unit/proxy and run the proxy-db shards from litellm-tests

* ci: fail the unit shard when circleci tests split errors

* test: drop restating comments from the gemini pass-through split

* build: point the local proxy unit targets at the nested tests/unit/proxy tree

* ci: exit the unit shard cleanly when circleci tests split assigns it no files

---------

Co-authored-by: yuneng <yuneng@berri.ai>
2026-09-24 22:59:11 +00:00