* test(pass-through): de-flake vertex spend-log assertion by re-billing
The vertex pass-through spend-log test asserted that a single billed
generateContent call moved the global spend aggregate within a fixed
wait. CI failures show the call returning a valid response with real
usage, yet spend never increasing over a 240s poll.
Pass-through spend logging is best-effort: the success handler is
enqueued on a background worker that can drop or time out an individual
event under load and never retries it, so one billed call occasionally
never reaches LiteLLM_SpendLogs. Waiting longer cannot recover a dropped
event; only re-issuing the call can.
Re-bill the call up to a few times and require at least one to be
tracked, mirroring the sibling jest test that already retries. The test
still fails hard if cost tracking is actually broken, since then every
call records nothing. Also sum spend across all returned days instead of
matching the runner's local 'today', removing a separate UTC-rollover
flake.
* test(pass-through): route vertex spend-log test through proxy via direct HTTP
The vertexai SDK, configured with location="global" and an http api_endpoint
override, intermittently sends generateContent to the public Vertex endpoint
instead of the proxy. Proxy logs from a failing run show all 46 of the test's
own spend-log polls reaching the proxy while zero generateContent calls did, so
LiteLLM never saw the billed call and no spend was ever recorded; re-billing
through the SDK could not help because every retry bypassed the proxy too.
Issue the pass-through request directly over HTTP so it always hits the proxy,
minting a Google token from the same service-account credentials, then assert
that the specific call's own spend log lands with spend > 0, a gemini model, and
custom_llm_provider vertex_ai. A small best-effort retry covers the rare case
where the background logging worker drops a single event; failing every attempt
still fails hard so the test keeps its teeth if cost tracking breaks.
* test(pass-through): reuse LITE_LLM_ENDPOINT and drop needless async in get_tracked_spend
test_basic_vertex_ai_pass_through_with_spendlog failed intermittently on
litellm_internal_staging (pipelines 82155, 82196, 82209, 82230) with "Spend
should be greater than before after 120s". Spend logging is async and batched,
so the pass-through call's cost sometimes had not landed within the 120s poll
window; one run ended on spend_after 0.0 because the final /global/spend/logs
read returned nothing and "or 0.0" recorded that as zero spend.
Widen the poll window to 240s and skip a transient empty read instead of
treating it as 0.0, so a momentary endpoint hiccup on the last poll no longer
fails an otherwise-billed call. The spend_after > spend_before assertion is
unchanged, so a genuinely unbilled call still fails the test
* Fix remaining VCR live-call leaks
* test(vcr): dedupe live-test helpers and drop spurious kwargs
Extract the duplicated isVertexQuotaError/runVertexRequestOrSkip Vertex
quota-skip helpers into tests/pass_through_tests/vertex_test_helpers.js and the
duplicated _skip_live_prompt_caching_test guard into tests/_live_test_helpers.py
so each lives in one place. In test_aarun_thread_litellm, build a separate
message_data carrying role/content for add_message and a thread_data without
them for run_thread/run_thread_stream/get_messages, which no longer receive the
spurious message fields.
* test(overhead): assert mock transport is exercised in non-streaming and stream tests
* test(pass-through): move Gemini pass-through tests to gemini-3.1-flash-lite
gemini-2.5-flash-lite is a generation behind and is slated for discontinuation on Vertex AI no earlier than October 16, 2026, so the pass-through suite was exercising an aging model. Every reference now points at gemini-3.1-flash-lite, which is GA and already priced in the cost map so the spend-logging assertions still compute a real cost
test_vertex.test.js also gains jest.retryTimes(3) to match the sibling spend tests. The CI failures were intermittent 429 RESOURCE_EXHAUSTED from Vertex quota pressure, and that file was the only one without a retry, so a single rate-limited request was failing the whole job
* test(pass-through): point Vertex tests at the global endpoint for gemini-3.1-flash-lite
gemini-3.1-flash-lite is not served on the Vertex us-central1 regional endpoint for the CI project, so the Vertex pass-through tests were returning a deterministic 404 "Publisher Model ... was not found or your project does not have access to it" while the Gemini API tests passed. Move the Vertex clients to the global location, which the pass-through router maps to aiplatform.googleapis.com, where the 3.1 family is served
`should_create_missing_views()` had `and result[0]["reltuples"]` which is
falsy when reltuples=0. On a fresh empty PostgreSQL table, CREATE INDEX sets
reltuples=0, causing the guard to return False and skip view creation entirely.
Views like MonthlyGlobalSpendPerKey are never created, and the
/global/spend/logs endpoint returns 500.
Fix: change to `and result[0]["reltuples"] is not None` so reltuples=0
(empty table) and reltuples=-1 (unanalyzed table) both correctly return True.
Also harden test_vertex_ai.py to return None instead of crashing with
JSONDecodeError when the spend-logs endpoint returns a non-JSON 500 response,
and add unit tests covering all three reltuples branches (0, -1, positive).
The test used a fixed 40s sleep before checking spend logs, but async
spend logging in CI sometimes takes longer to flush. Replace with a
polling loop (10s interval, 120s max) that exits early on success.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix 1.1: Make ResponseApplyPatchToolCall import conditional with try/except
for compatibility with openai==1.100.1 (CI environment)
Fix 1.2: Move Router creation inside mock context in vector store tests
so mocks are applied before Router captures function references
Fix 1.3: Update test_model_group_info_e2e to check for 'anthropic/*'
wildcard group instead of specific model names not in proxy config
Fix 2.1: Increase redis cache test sleep from 1s to 5s
Fix 2.2: Increase spend accuracy test sleep from 25s to 45s
Fix 2.3: Add 0.5s sleep between budget test calls
Fix 2.4: Increase vertex AI spend test sleep from 20s to 40s
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
* feat - allow tagging vertex JS SDK request
* add unit testing for passing headers for pass through endpoints
* fix allow using vertex_ai as the primary way for pass through vertex endpoints
* docs on vertex js pass tags
* add e2e test for vertex pass through with spend tags
* add e2e tests for streaming vertex JS with tags
* fix vertex ai testing
* use 1 file for AnthropicPassthroughLoggingHandler
* add support for anthropic streaming usage tracking
* ci/cd run again
* fix - add real streaming for anthropic pass through
* remove unused function stream_response
* working anthropic streaming logging
* fix code quality
* fix use 1 file for vertex success handler
* use helper for _handle_logging_vertex_collected_chunks
* enforce vertex streaming to use sse for streaming
* test test_basic_vertex_ai_pass_through_streaming_with_spendlog
* fix type hints
* add comment
* fix linting
* add pass through logging unit testing
* fix(model_prices_and_context_window.json): add cost tracking for more vertex llama3.1 model
8b and 70b models
* fix(proxy/utils.py): handle data being none on pre-call hooks
* fix(proxy/): create views on initial proxy startup
fixes base case, where user starts proxy for first time
Fixes https://github.com/BerriAI/litellm/issues/5756
* build(config.yml): fix vertex version for test
* feat(ui/): support enabling/disabling slack alerting
Allows admin to turn on/off slack alerting through ui
* feat(rerank/main.py): support langfuse logging
* fix(proxy/utils.py): fix linting errors
* fix(langfuse.py): log clean metadata
* test(tests): replace deprecated openai model
