* test(ci): serve /moderations from the canned OpenAI mock
The otel proxy E2E job points its `openai/*` wildcard deployment at the
canned mock, and #37492 made `get_model_list` agree with
`get_available_deployment` on bare model names. /moderations now resolves
`omni-moderation-latest` to that wildcard deployment the way
/chat/completions already did, so the request lands on the mock, which
never implemented the route and answers a bare 404.
Add /moderations and /v1/moderations to the mock, returning an
OpenAI-shaped response with one result per input item.
* style(ci): annotate the new moderations locals as Final
* test: point router/completion/triton tests at the local fake OpenAI endpoint
The shared Railway-hosted mock (exampleopenaiendpoint-production.up.railway.app)
takes down unrelated CI jobs whenever it is unreachable. #30695 moved the mounted
proxy configs onto a job-local fake server but left these in-Python api_base
literals pointing at the dead host, so litellm_router_testing, local_testing_part1,
local_testing_part2 and llm_translation_testing still fail with a 404
"Application not found" when Railway is down
Resolve the api_base from FAKE_OPENAI_API_BASE (default http://127.0.0.1:8190)
through a shared helper, auto-start the canned server from the local_testing and
llm_translation conftests when nothing is already serving, and extend the server
with a Triton embeddings route and a slow-endpoint delay so the triton and
latency-timeout tests run fully offline. The deliberately broken fallback URL is
left as-is so fallback handling still has a failing upstream
* fix: ignore non-loopback FAKE_OPENAI_API_BASE so the local mock is used in CI
* fix: drop 0.0.0.0 from loopback hosts, an unreliable client connect target
* fix(tests): keep fake OpenAI mock alive across xdist workers
ensure_fake_openai_endpoint registered atexit on the worker that spawned
the subprocess, so under -n 4 the first worker to drain its queue would
terminate the shared mock while siblings were still hitting it. Detach
the child via start_new_session and drop the per-worker teardown; reuse
on /health handles re-runs and CI containers clean up themselves
Several CI jobs run the proxy against a model whose api_base is a shared
"fake OpenAI endpoint" hosted on Railway
(exampleopenaiendpoint-production.up.railway.app) so the E2E runs return
canned responses without paying for or depending on a live provider. When
that single deployment is down, every one of those jobs fails with
"404 Application not found" even though nothing in the PR is broken; the
whole repo is coupled to the uptime of one free external service.
This adds tests/_fake_openai_endpoint_server.py, a small canned-response
OpenAI-shaped server (chat, text, embeddings, streaming with usage, and the
"429" rate-limit special case), and a reusable start_fake_openai_endpoint
CircleCI command that runs it on host port 8190 and waits until healthy. The
affected jobs now inject FAKE_OPENAI_API_BASE pointing at the local server,
and the example configs they mount resolve api_base from that env var. The
intentionally bad fallback URL in proxy_server_config.yaml is left untouched
so the fallback test still exercises a failing upstream.
Wired into build_and_test, litellm_router_testing,
db_migration_disable_update_check, proxy_logging_guardrails_model_info_tests,
proxy_spend_accuracy_tests, proxy_multi_instance_tests,
proxy_store_model_in_db_tests, and proxy_build_from_pip_tests.
