litellm/tests
devin-ai-integration[bot] 075655c7ee
test(azure_sentinel): pin batch_size as a per-request bound under concurrent events (#40320)
* test(azure_sentinel): pin batch_size as a per-request bound under concurrent events

Adds a regression test to the mapped Azure Sentinel test file for the concurrency scenario from LIT-6920: 40 records logged concurrently at batch_size=5 while each ingestion request is still in flight. Asserts no request carries more than batch_size records, every record arrives exactly once in order, and the queue is empty afterwards. Runs for both the standard log queue and the audit log queue.

The test fails on the tree before #39880 (whole shared queue serialized per threshold send, then cleared) and passes on current staging. It is independent of the size-split coverage that #39880 added for LIT-5899.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(azure_sentinel): gate the first send on events so later records provably arrive while it is in flight

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-08 16:55:40 -07:00
..
agent_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
audio_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
base_sdk_tests
basic_proxy_startup_tests
batches_tests feat(batches): enrich batch cost rows with breakdown, identity, session, and org spend 2026-09-03 17:23:53 -04:00
benchmarks
code_coverage_tests fix(ci): keep an expression matrix directive out of the comparison 2026-09-06 02:59:16 -07:00
documentation_tests Merge remote-tracking branch 'origin/main' into litellm_bedrock_messages_disconnect_billing 2026-08-31 08:58:37 -07:00
e2e Merge pull request #40041 from BerriAI/litellm_presidio_ui_user_story_e2e 2026-09-07 15:41:33 -07:00
enterprise fix(batches): register ownership for every batch create path (#39810) 2026-09-04 23:59:51 -07:00
guardrails_tests fix(logging): blocked requests no longer report guardrail_status=success in multi-guardrail configs (#39596) 2026-09-03 17:33:31 -07:00
image_gen_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
integration test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
litellm-proxy-extras test(proxy-extras): fake run_prisma instead of subprocess.run in the migrate deploy harness 2026-09-03 16:10:44 -07:00
litellm_utils_tests test(aiohttp): pin NO_PROXY so proxy env cannot hijack the refused-port probe 2026-08-28 12:45:46 -07:00
llm_responses_api_testing fix(guardrails): deliver modify_response block as valid SSE on streaming chat and Responses 2026-08-31 16:01:39 -07:00
llm_translation test: collapse blank lines left by removed tests 2026-09-08 02:21:55 +00:00
load_tests feat(proxy): per-worker admission control that rejects excess requests with 503 (#39352) 2026-09-03 18:19:04 -07:00
local_testing chore: merge litellm_internal_staging into litellm_lit_7039_least_busy_shared_counts 2026-09-08 17:11:05 +00:00
logging_callback_tests Merge branch 'litellm_internal_staging' into litellm_fix_failing_request_slowdown 2026-09-03 00:20:51 -07:00
mcp_tests fix(mcp): scope allow-all servers to virtual keys (#39531) 2026-09-03 10:32:03 -07:00
multi_instance_e2e_tests test: say whether a match= pattern is a regex or a literal (ruff RUF043) 2026-08-21 16:25:33 -07:00
ocr_tests fix(vertex): avoid duplicate DeepSeek OCR model namespace 2026-09-01 14:39:15 -07:00
openai_endpoints_tests refactor(tests): assign the streamed id and lock poll once instead of rebinding 2026-09-03 13:49:43 -07:00
otel_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
pass_through_tests fix(logging): key bridged /v1/messages rows on the id the caller received 2026-09-03 01:15:06 -07:00
pass_through_unit_tests test(websearch): carry a reasoned test-quality suppression on the router patch 2026-08-31 22:32:05 -07:00
proxy_admin_ui_tests refactor(tests): assign the streamed id and lock poll once instead of rebinding 2026-09-03 13:49:43 -07:00
proxy_behavior feat(ui): itemize auto-router classification spend (#40168) 2026-09-07 16:29:19 -07:00
proxy_e2e_anthropic_messages_tests
proxy_migration_tests chore: merge litellm_internal_staging into litellm_fix_invalid_index_after_migration_deadlock 2026-09-08 17:07:34 +00:00
proxy_security_tests
proxy_unit_tests Merge pull request #39626 from BerriAI/litellm_batch_ui_logs 2026-09-08 15:19:25 -07:00
router_unit_tests fix(router): count allowed_fails in the shared router cache so multi-worker proxies bench a deployment fleet-wide (#40224) 2026-09-08 13:54:48 -07:00
rust-python-harness test(ocr): add SDK callback E2E parity (#40061) 2026-09-07 11:23:12 -07:00
search_tests fix(search): harden bing_grounding auth, result cap, status, and cost 2026-08-24 12:26:07 -07:00
spend_tracking_tests
store_model_in_db_tests test(store_model_in_db): accept both 400 shapes in the unknown-model spend log test 2026-09-04 18:52:17 -07:00
test_litellm test(azure_sentinel): pin batch_size as a per-request bound under concurrent events (#40320) 2026-09-08 16:55:40 -07:00
test_litellm_rust test: add Rust extension pytest contract (#40181) 2026-09-07 18:46:29 -07:00
unified_google_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
vector_store_tests fix(vector-store): carry request metadata into the Router executor built from the router kwarg 2026-09-02 16:53:05 -07:00
windows_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
__init__.py
_fake_openai_endpoint_server.py test(timeout): time out against the local fake endpoint instead of api.openai.com 2026-09-03 09:53:30 -07:00
_flush_vcr_cache.py
_live_test_helpers.py
_openai_record_replay_proxy.py
_vcr_conftest_common.py
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD
test_anthropic_compaction_usage.py
test_budget_management.py
test_callbacks_on_proxy.py
test_debug_warning.py
test_default_encoding_non_root.py
test_end_users.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py
test_health.py
test_keys.py test: fix staging CI regressions from #38182, #38144, #38265, #37962, and #37969 2026-08-25 23:01:20 -07:00
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py test: address review notes on the chronic-test repairs 2026-09-04 10:18:15 -07:00
test_new_vector_store_endpoints.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_openai_endpoints.py
test_organizations.py
test_otel_thread_leak.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_resource_cleanup.py
test_rust_python_harness.py test: address review notes on the chronic-test repairs 2026-09-04 10:18:15 -07:00
test_service_logger_otel.py
test_spend_logs.py
test_team.py
test_team_logging.py
test_team_members.py test: reject assertions on a caught error inside except (ruff PT017) 2026-08-21 13:35:08 -07:00
test_users.py

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.