* test(e2e): OTEL trace completeness on /chat/completions against a local Jaeger destination
Adds the logging-suite infrastructure for LIT-3787 trace-completeness coverage:
a jaeger service in the compose stack as the OTEL v2 destination (arize_phoenix
preset pointed at it via PHOENIX_COLLECTOR_HTTP_ENDPOINT, so gen-AI spans export
through a preset-owned provider - the code path where trace splits happen), a
typed Jaeger query read-back client, and the first test: one successful
non-streaming /chat/completions call exports ONE complete trace (root SERVER
span + auth/db/cost children + gen-AI CLIENT span, no dangling parents).
* test(e2e): harden the otel trace read-back per review
Jaeger reads now query server-side by the litellm.call_id span tag instead of
paging recent traces and filtering client-side; the compose stack's background
jobs alone can push a request trace past the page. A failed query hard-fails
instead of reading as an empty result, the settle predicate now also waits for
the prefix-matched db span the assertion demands, parent-chain walking follows
CHILD_OF references only, the zero-trace and split-trace failures get distinct
messages, jaeger gets a healthcheck so the depends_on condition is accurate,
and the chat docstring names the route the code actually asserts
* test(e2e): author the chat trace test docstring
* Update logging section in CLAUDE.md
Removed mention of OTEL trace-tree completeness from logging integration section.
* fix(e2e): wire batch provider secrets for docker and k8s
Point batch deployments at the credential field names and os.environ refs
the gateway actually resolves from process env (compose .env or EKS secret
mounts). Missing secrets skip instead of failing red so a red run means a
product bug. Mirror S3 bucket env aliases in docker-compose for provider_fallback
* fix(e2e): drop batch provider_env unit tests
The batches suite is live e2e only; no monkeypatch or unit-level tests
* fix: batch credentials, provider list, and team db lookup
Keep object-storage fields through CredentialLiteLLMParams and resolve
os.environ/ refs when reading deployment credentials so Vertex/Bedrock
batch file uploads see bucket and AWS keys from K8s/docker env
Skip managed batch list when the request is provider-scoped so
/{provider}/v1/batches list works instead of 500
Force DB on check_db_only team lookups and stop masking non-404 errors
as "team doesn't exist"
Drop e2e runner-side skip helpers; hard-fail on missing gateway secrets
* fix: tag reseed, team window spend, and remaining e2e flakes
Reseed spend:tag counters from LiteLLM_TagTable so cold redis still
enforces after the spend writer flushes
When applying post-call cost to team multi-window counters, load the
team from the DB if it is missing from the management cache so window
spend is not dropped on cache misses
Harden cold-counter reseed e2e (namespace-aware keys, burst success,
poll). Give tag budget more headroom. Retry /key/update on redis DNS
blips. Ensure NLTK punkt_tab is present for pipecat realtime audio
* revert: drop product code changes; e2e-only scope
Reverts all litellm/ and unit-test product edits. This branch is limited
to tests/e2e per contributor instruction
* fix(e2e): harden batch list and team member setup races
provider_fallback list falls back when managed batches reject provider
filtering. Team create waits for /team/info and member_add retries on
transient team-not-found so split control-plane lag does not red the suite
* fix(e2e): remove .env.example
Leave local .env and docker-compose env wiring as the secret source
* fix(e2e): wire files_settings and faster budget rescheduler for compose
OpenAI/Azure batch file uploads need files_settings; budget reset e2e needs a
short rescheduler window. Drop unsupported bedrock-encoded create_batch cells,
tolerate bedrock file.bytes=0, and surface team-info wait failures instead of
hanging silently
* chore(e2e): strip verbose comments from batch capabilities
* fix(e2e): assert managed list fallback before provider_fallback skip
When provider-scoped list is rejected, still fetch the unfiltered list and
check the envelope. Only skip membership when the id is a raw
provider_fallback batch that managed list cannot index
* fix: rust ocr tests finally pass
* fix: move realtime dir
* fix(realtime): normalize azure realtime api_base to host for Foundry endpoints
The azure realtime handler appended the realtime path to api_base verbatim, so a
Foundry base carrying a project path (.../api/projects/<name>) produced an invalid
realtime URL and the websocket handshake hung. Normalize api_base to scheme and host
before building the realtime path so both Azure OpenAI and Foundry bases connect
Point the e2e realtime azure deployment at the GA gpt-realtime model and stop passing
the os.environ refs the realtime path never unwraps, resolving them from the gateway
env by name instead. Drop the local docker-compose scaffolding from the tree
* test(e2e): add Gateway.list_files and list_fine_tuning_jobs for the discovery suite
The discovery endpoints suite calls client.gateway.list_files and
list_fine_tuning_jobs, which did not exist on Gateway, so both tests errored with
AttributeError before reaching the proxy. Add the two GET wrappers using the
existing FileListResponse / FineTuningJobsResponse models
* revert(realtime): drop azure realtime api_base host-normalization
The azure realtime handshake failure was a config issue, not a litellm bug: the
realtime base was set to the Azure AI Foundry project endpoint (.../api/projects/<p>),
but the OpenAI-compatible realtime route lives at the resource root. litellm correctly
appends the realtime path to whatever base it is given, so pointing the realtime
deployment at the resource root is the fix and no core change is needed
* fix(ocr): route azure_ai doc-intelligence to its own endpoint at the source
get_llm_provider inherits AZURE_AI_API_BASE into api_base for every azure_ai/* OCR
model, but Azure Document Intelligence is a separate resource reached via
AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT, so doc-intelligence requests went to the wrong
host. Stop inheriting the azure_ai base for doc-intelligence models so api_base stays
unset and both the rust bridge and the python get_complete_url fall back to the
document-intelligence endpoint. This drops the earlier _rust_bridge_api_base reorder,
which only covered the rust path and let the env silently override an explicit api_base
* refactor(ocr): consolidate azure doc-intelligence detection; keep explicit api_base
Extract is_azure_document_intelligence_model as the single source of truth for the azure_ai doc-intelligence sub-route so the check is no longer duplicated across _prepare_ocr_request and _rust_bridge_api_base, and gate the dynamic_api_base suppression on the caller not supplying an api_base so an explicit endpoint is always honoured. Restore xai to the realtime PROVIDERS as a documented disabled entry instead of dropping it silently, and add a regression test pinning doc-intelligence api_base resolution.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): scope key models dropdown options to the key's team
A teamless key no longer offers the all-team-models option in the create and
edit forms; the backend expands that sentinel to the full proxy model list when
no team is attached, which is rarely what the user intended. A team key no
longer surfaces the all-proxy-models sentinel that leaks in verbatim when the
team's own model list carries it; the dropdown keeps All Team Models plus the
team's individual models.
Adds browser coverage to the management e2e suite: playwright (an optional
dependency behind importorskip) drives the proxy-served dashboard at /ui,
asserts the dropdown options a real user sees for teamless and team keys on
both create and edit, and walks the create modal end to end, reading the
persisted key back through /key/info.
* fix(ui): offer all-proxy-models on teamless keys in the models dropdown
A teamless key has no team allowlist to inherit, so the dropdown now offers All
Proxy Models in place of All Team Models on both the create and edit forms, with
the same exclusive-selection handling. Component and browser e2e tests updated to
pin the swapped option pair; the teamless create case now also walks the modal end
to end and reads the persisted key back through /key/info.
* test(ui): update no-team key creation spec to pick All Proxy Models
The create modal no longer offers All Team Models without a team; the teamless
path now offers All Proxy Models, which is what this spec exercises
* fix(ui): gate All Team Models on the team object being loaded
When a key has a team_id but the teams prop does not yet include the matching team, availableModels stays empty and the models dropdown rendered All Team Models on its own with nothing to compare against. Gate the option on the team object being present so it only appears once team models are known, and add a regression test for the loading state
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): filter all-proxy-models from teamless model fetch in key edit form
The teamless fetch path stored modelAvailableCall results without excludeProxyWideSentinel, so an all-proxy-models entry in the response rendered a second option colliding with the hardcoded All Proxy Models sentinel. Apply the same filter used on the team path and add a regression test asserting the sentinel option is not duplicated
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(e2e): untrack gateway config and document e2e test location
Stop tracking tests/e2e/gateway/litellm-config.yml so the local proxy config stays on the machine
Add a note to CLAUDE.md that new e2e tests belong in tests/e2e/ and must follow that directory's conventions
* chore(e2e): add self-contained docker compose stack for local runs
Ship a docker-compose.yml that starts the proxy with a throwaway Postgres and Redis and inlines the proxy config with example models, so contributors can bring up a local gateway with nothing but a .env. Update CONTRIBUTING.md to match the inline-config flow
* chore(e2e): drop the second gemini deployment; one key is enough locally
* docs(e2e): make pre-commit steps ordered and require flagging internally found issues
