mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
a7d8c6f467
* test(pass-through): de-flake vertex spend-log assertion by re-billing The vertex pass-through spend-log test asserted that a single billed generateContent call moved the global spend aggregate within a fixed wait. CI failures show the call returning a valid response with real usage, yet spend never increasing over a 240s poll. Pass-through spend logging is best-effort: the success handler is enqueued on a background worker that can drop or time out an individual event under load and never retries it, so one billed call occasionally never reaches LiteLLM_SpendLogs. Waiting longer cannot recover a dropped event; only re-issuing the call can. Re-bill the call up to a few times and require at least one to be tracked, mirroring the sibling jest test that already retries. The test still fails hard if cost tracking is actually broken, since then every call records nothing. Also sum spend across all returned days instead of matching the runner's local 'today', removing a separate UTC-rollover flake. * test(pass-through): route vertex spend-log test through proxy via direct HTTP The vertexai SDK, configured with location="global" and an http api_endpoint override, intermittently sends generateContent to the public Vertex endpoint instead of the proxy. Proxy logs from a failing run show all 46 of the test's own spend-log polls reaching the proxy while zero generateContent calls did, so LiteLLM never saw the billed call and no spend was ever recorded; re-billing through the SDK could not help because every retry bypassed the proxy too. Issue the pass-through request directly over HTTP so it always hits the proxy, minting a Google token from the same service-account credentials, then assert that the specific call's own spend log lands with spend > 0, a gemini model, and custom_llm_provider vertex_ai. A small best-effort retry covers the rare case where the background logging worker drops a single event; failing every attempt still fails hard so the test keeps its teeth if cost tracking breaks. * test(pass-through): reuse LITE_LLM_ENDPOINT and drop needless async in get_tracked_spend |
||
|---|---|---|
| .. | ||
| ruby_passthrough_tests | ||
| base_anthropic_messages_test.py | ||
| package-lock.json | ||
| package.json | ||
| test_anthropic_passthrough.py | ||
| test_anthropic_passthrough_basic.py | ||
| test_anthropic_passthrough_python_sdkpy | ||
| test_assembly_ai.py | ||
| test_gemini.js | ||
| test_gemini_with_spend.test.js | ||
| test_hosted_vllm_passthrough.py | ||
| test_local_gemini.js | ||
| test_local_vertex.js | ||
| test_mcp_routes.py | ||
| test_openai_assistants_passthrough.py | ||
| test_vertex.test.js | ||
| test_vertex_ai.py | ||
| test_vertex_with_spend.test.js | ||
| vertex_key.json | ||
| vertex_test_helpers.js | ||