scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-16 04:29:31 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 138
litellm/tests/integration/_support
History
yuneng-jiang 46913be92b
test(integration): wait on the right signal in five flaky integration contracts (#46737) 
* test(integration): wait on the right signal in five flaky integration contracts

The Prometheus shared-label tests waited for the latency metric and then read the
request counter, which the logger writes after an await. They now wait for the counter

The auto-router session poll returned on the first 200 while the caller expected two
turns, so it now waits for the expected turn count

The S3 queue cap test let its failing sink recover on a 15s clock that could expire
mid-flush. The sink now stays down until a flush fails after every event is queued

K3 normally takes 168-175s against a 180s limit, so the limit is now 300s

The MCP env var browser spec reused an idle keep-alive socket that the proxy closes
after 5s, so its API calls now send Connection: close

* test(integration): add until_asserted so a wait polls the exact asserts it guards

eventually() takes a predicate separate from the asserts that follow it, which lets a
test wait on one value and then assert on another written later. until_asserted reruns
a block of asserts until they all pass and re-raises the last failure on timeout

The Prometheus shared-label tests and the auto-router session check now use it
2026-10-15 13:48:51 -07:00
..
__init__.py test: add CircleCI integration contract foundation 2026-09-14 03:30:52 -07:00
actors.py test: migrate tests/proxy_behavior into integration and unit tiers (#45939) 2026-10-11 08:28:42 -07:00
agentic_probe.py fix(responses): honor caller stream flag when provider forces SSE (internal copy of #34095) (#41235) 2026-10-06 03:30:47 +00:00
anthropic_refusal.py fix(anthropic): keep refusal stop_reason and stop_details on the logged response (#46104) 2026-10-14 14:42:49 -07:00
anthropic_sse.py fix(router): retry a /v1/messages stream the provider drops before the first content chunk (#44276) 2026-10-05 22:53:39 +00:00
anthropic_thinking.py test: expect the list-position index on streamed thinking blocks in the integration suite (#46618) 2026-10-15 05:52:56 +00:00
asgi.py fix(ci): isolate integration workers and repair test fixtures (#45863) 2026-10-11 01:56:02 -07:00
async_server.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
aws_control_plane.py fix(bedrock): list the account's invocable models behind bedrock/* when check_provider_endpoint is on (#43981) 2026-10-09 14:14:17 -07:00
bedrock_discovery.py fix(bedrock): list the account's invocable models behind bedrock/* when check_provider_endpoint is on (#43981) 2026-10-09 14:14:17 -07:00
bedrock_runtime_peer.py test(ci): fix six CircleCI test regressions on main (#44429) 2026-10-03 20:55:01 -07:00
browser_state.py test: add extension and browser integration contracts 2026-09-16 13:15:53 -07:00
budget.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
claude_code.py test(anthropic): native /v1/messages reasoning integration tests built on a captured Claude Code request (#43361) 2026-10-01 09:12:53 -07:00
client.py test(integration): wait on the right signal in five flaky integration contracts (#46737) 2026-10-15 13:48:51 -07:00
codex_vendor.py fix(responses): honor caller stream flag when provider forces SSE (internal copy of #34095) (#41235) 2026-10-06 03:30:47 +00:00
daily_activity.py fix(proxy): look up hashed key names with two spend log rows per key (#43656) 2026-09-29 20:21:55 -07:00
daily_spend_rows.py fix(ui): group cost optimization cache leakage by model group (#43008) 2026-09-24 23:51:51 -07:00
database.py test(integration): isolate every multi-worker integration suite by default (#46027) 2026-10-12 03:46:06 -07:00
database_relay.py fix(scim): write a group's roster in one locked transaction instead of one member at a time (#45197) 2026-10-15 10:57:00 -07:00
device_login.py fix(chatgpt,github_copilot): refuse device-code login inside an event loop or worker thread (#39585) 2026-10-06 03:20:07 +00:00
forward_proxy.py feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN (#45678) 2026-10-09 22:51:19 -07:00
generation.py fix(tests): resolve the integration support package without run.py's PYTHONPATH 2026-09-15 21:10:08 -07:00
idle_gateway.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
interception_vendor.py fix(websearch_interception): drop stream_options when forcing stream=False and replay it on the final stream (#45433) 2026-10-09 22:16:02 -07:00
mail.py feat(proxy): email alerts at configured percentages of a team member budget (#42665) 2026-09-26 02:42:45 +00:00
manifest.py fix(caching): never store or serve a chat completion with no choices (#44709) 2026-10-06 20:07:30 +00:00
mcp.py test: migrate proxy security, schema migration and MCP legacy tests (#45937) 2026-10-11 16:17:24 -07:00
mcp_e2e_upstream_server.py test: migrate proxy security, schema migration and MCP legacy tests (#45937) 2026-10-11 16:17:24 -07:00
mcp_grants.py fix(mcp): resolve team-granted toolsets for non-admin keys and dashboard sessions (#43908) 2026-10-01 14:15:06 -07:00
mcp_stdio_peer.py test(integration): add MCP gateway coverage wave 1 with a dedicated mcp shard and proxy coverage artifact (#42711) 2026-09-23 07:48:46 -07:00
oauth_server.py test(integration): credential canary slots for MCP and pass-through credentials (#43308) 2026-09-28 17:10:51 -07:00
openai_wire.py test: fix shared-provider discovery, Codex catalog size and generated master key mismatches in CircleCI suites (#44905) 2026-10-07 00:11:25 +00:00
ordering.py fix(ci): isolate integration workers and repair test fixtures (#45863) 2026-10-11 01:56:02 -07:00
otlp_sink.py fix(ci): isolate integration workers and repair test fixtures (#45863) 2026-10-11 01:56:02 -07:00
pdf_document.py fix(token_counter): price a base64 PDF document per page instead of as one image (#45301) 2026-10-08 21:20:46 -07:00
prisma_clients.py test: migrate tests/proxy_behavior into integration and unit tiers (#45939) 2026-10-11 08:28:42 -07:00
process.py fix(scim): write a group's roster in one locked transaction instead of one member at a time (#45197) 2026-10-15 10:57:00 -07:00
prometheus_series.py feat(prometheus): cap series per metric for every labeled metric (#44420) 2026-10-06 01:28:44 +00:00
prompt_cache_breakpoint.py fix(responses): map input_audio blocks in the chat-to-Responses bridge (#45224) 2026-10-07 22:25:35 -07:00
provider.py fix(ci): isolate integration workers and repair test fixtures (#45863) 2026-10-11 01:56:02 -07:00
proxy.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
redaction_probe.py fix(logging): redact nested callback input and provider response fields (#45923) 2026-10-13 12:25:56 -07:00
redis_breaker_worker.py fix(caching): report Redis circuit-breaker metrics through the Prometheus services logger (#46484) 2026-10-14 14:29:54 -07:00
redis_process.py test(integration): cut CircleCI proxy boot time with concurrent worker stacks and single-worker test proxies (#46037) 2026-10-12 09:48:42 -07:00
responses_stream.py fix(responses): keep the provider code and message on top-level stream error events (#46438) 2026-10-14 16:16:46 -07:00
responses_vendor.py fix(responses): drop bridge-minted reasoning items from OpenAI replays (#44132) 2026-10-02 19:39:48 -07:00
routing.py test(integration): isolate every multi-worker integration suite by default (#46027) 2026-10-12 03:46:06 -07:00
runtime.py test(integration): isolate shared proxy state behind six order-dependent flakes (#46172) 2026-10-13 09:55:11 -07:00
safeguards_rig.py feat(anthropic): generate auto mode safeguard_results with a classifier model on the /v1/messages chat bridge (#45600) 2026-10-14 13:09:11 +00:00
sagemaker_embedding.py feat(sagemaker)!: inference components and OpenAI-compatible embeddings (#44985) 2026-10-12 22:25:52 +00:00
sharding.py perf(ci): use native timing splits with bounded integration concurrency (#45837) 2026-10-10 13:35:11 -07:00
sigv4.py fix(bedrock): list the account's invocable models behind bedrock/* when check_provider_endpoint is on (#43981) 2026-10-09 14:14:17 -07:00
timing.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
tls.py fix(azure_storage): name Data Lake objects without base64 padding or slashes (#43914) 2026-09-30 18:17:13 -07:00
tool_rows.py feat(tool-policies): show the user who owns the key that discovered a tool (#43892) 2026-10-01 20:34:57 +00:00
tool_search.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
upstream.py test(cost): cover custom deployment aliases, off-peak pricing, bridge usage shapes and duplicate spend rows (#46708) 2026-10-15 10:44:51 -07:00
vertex.py fix(vertex_ai): forward system and tools to partner model count_tokens (#43900) 2026-10-03 16:36:14 -07:00
wire.py fix(streaming): add CustomStreamWrapper.close() and release the connection when a sync stream ends (#46354) 2026-10-14 19:14:04 +00:00
worker_readiness.py test(integration): cover responses endpoint contracts (#44733) 2026-10-14 06:59:15 +00:00
workers.py test(integration): cut CircleCI proxy boot time with concurrent worker stacks and single-worker test proxies (#46037) 2026-10-12 09:48:42 -07:00