Two unrelated causes, both leaving staging red with tests that no longer describe
anything true.
#38264 gave LangFuseLogger a langfuse_environment argument and started carrying it
in the credentials dict. The handler test's fake logger did not accept the new
keyword, so constructing it raised TypeError, and four cases in
test_langfuse_unit_tests rebuilt the cache key by hand from three fields and missed
on the four-field key production now writes. Caching itself was never broken: the
handler sets and gets with the same dict. The fake now takes the argument and
asserts it is forwarded, and the cache assertion issues a second identical request
and expects the same logger back, which is the behaviour that matters and cannot
rot the next time a credential field is added.
OpenAI has retired the Assistants API. /v1/assistants and /v1/threads both answer
404 with a valid key, where every live route answers 401, so nothing calling them
can pass again. test_custom_logger_passthrough covered generic passthrough logging
and only used assistants because it is a route with no provider-specific handler;
it moves to /v1/moderations, which is still unclaimed by
_is_supported_openai_endpoint, so the same generic branch is exercised. The two
tests there asserted the same thing against different dead routes, so they collapse
into one. The Ruby suite existed solely to drive assistants, threads, messages and
runs, so it goes along with the RVM and bundler steps that were installed only to
run it, and the two dead OpenAI assistants cases leave
test_openai_assistants_passthrough.
The Azure assistants case in that file stays. Azure runs its own lifecycle and I
could not reach the CI deployment to check whether that API is still there.
* test(proxy): poll for image-gen spend instead of a fixed 5s sleep
test_key_info_spend_values_image_generation failed once on litellm_internal_staging
(pipeline 82282) with "spend did not increase on an identical repeat image call"
(assert 0.24966 > 0.24966). The test made the second image call, slept 5s, then
read the key's spend once. Response caching is commented out in
proxy_server_config.yaml and no sibling test enables it, so the likely cause is
async/batched spend logging not having flushed the repeat call's cost within 5s,
which the build_and_test job aggravates by running every tests/test_*.py against
one shared proxy under pytest -n 4.
Poll the key's spend for up to 60s and break as soon as it grows. This removes
the timing flake while preserving the canary: if the repeat were genuinely
unbilled (for example the proxy response cache being on), spend never grows, the
poll times out, and the assertion still fails.
* test(pass_through): raise ruby assistants client request_timeout to 600s
The streaming assistants example in openai_assistants_passthrough_spec.rb hit
Net::ReadTimeout on litellm_internal_staging (pipeline 82280), failing at roughly
125s which is ruby-openai's default request_timeout of 120s. An assistants run
with the code_interpreter tool can occasionally take longer than that to stream
its first content back through the pass-through.
Raise the client's request_timeout to 600s, matching the 600s timeout the Python
pass-through e2e tests already use, so a slow-but-healthy streaming run no longer
trips the default read timeout.
