mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-13 03:59:12 +00:00
02feb67c20
* test(integration): build worker stacks concurrently and record proxy boot phases The controller built every xdist worker's database, Redis, upstream and gateway proxy one after another before any test ran, about 95 s per extensions shard, and since every multi-worker suite is now isolated the same serial wait hits mcp, security and cost too. The stacks now start on a thread pool bounded by the CPU count. Each stack reads its endpoints and results directory from an explicit environment instead of a patched os.environ, every stack that started is registered for cleanup, and a failure in one stops the others before the error is raised Owned proxy startup.json now records the wall-clock spawn time, the worker count and when the Prisma CLI child ran, when the port started accepting and when the first worker answered readiness, and exit.json records when the stop started and ended, so the boot time can be split and overlapping gateway boots can be seen in the artifacts * test(integration): boot one uvicorn worker where a test does not assert cross-worker behaviour A 2-worker owned proxy takes about twice as long to become ready as a 1-worker one, because the uvicorn parent imports the app and each spawned worker imports it again. The passthrough upstream error visibility cases, the single-request S3 dedupe cases and the single-request spend log payload cases send one or two requests and assert on the response, the proxy log and the spend row, none of which depends on a second worker, so they now boot the default single worker Cases that do rely on two workers keep them, the concurrent S3 burst through a sink outage, previous_response_id replay, response cache hits, the concurrent spend bursts and upstream stop and recovery * test(observability): settle OTel excluded-services traces by a quiet period instead of a full window _trace_spans slept through its whole 15 or 30 s window on every call even when the trace had long stopped growing. It now polls and returns once the trace has spans and has not gained one for five tenant export intervals, and the module pins the tenant batch exporter's schedule delay to 1 s so that bound is set by the test rather than by the SDK default. Absence checks still run only after the trace stopped growing, and against a build that leaks postgres spans to the tenant they still fail These cases also boot one uvicorn worker now. None of them asserts anything about a second worker --------- Co-authored-by: Yuneng Jiang <yuneng@berri.ai> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| actors.py | ||
| agentic_probe.py | ||
| anthropic_sse.py | ||
| anthropic_thinking.py | ||
| asgi.py | ||
| async_server.py | ||
| aws_control_plane.py | ||
| bedrock_discovery.py | ||
| bedrock_runtime_peer.py | ||
| browser_state.py | ||
| budget.py | ||
| claude_code.py | ||
| client.py | ||
| codex_vendor.py | ||
| daily_activity.py | ||
| daily_spend_rows.py | ||
| database.py | ||
| database_relay.py | ||
| device_login.py | ||
| forward_proxy.py | ||
| generation.py | ||
| interception_vendor.py | ||
| mail.py | ||
| manifest.py | ||
| mcp.py | ||
| mcp_e2e_upstream_server.py | ||
| mcp_grants.py | ||
| mcp_stdio_peer.py | ||
| oauth_server.py | ||
| openai_wire.py | ||
| ordering.py | ||
| otlp_sink.py | ||
| pdf_document.py | ||
| prisma_clients.py | ||
| process.py | ||
| prometheus_series.py | ||
| prompt_cache_breakpoint.py | ||
| provider.py | ||
| proxy.py | ||
| redis_process.py | ||
| responses_stream.py | ||
| responses_vendor.py | ||
| routing.py | ||
| runtime.py | ||
| sharding.py | ||
| sigv4.py | ||
| timing.py | ||
| tls.py | ||
| tool_rows.py | ||
| upstream.py | ||
| vertex.py | ||
| wire.py | ||
| workers.py | ||