devin-ai-integration[bot] 075655c7ee test(azure_sentinel): pin batch_size as a per-request bound under concurrent events (#40320) ... * test(azure_sentinel): pin batch_size as a per-request bound under concurrent events Adds a regression test to the mapped Azure Sentinel test file for the concurrency scenario from LIT-6920: 40 records logged concurrently at batch_size=5 while each ingestion request is still in flight. Asserts no request carries more than batch_size records, every record arrives exactly once in order, and the queue is empty afterwards. Runs for both the standard log queue and the audit log queue. The test fails on the tree before #39880 (whole shared queue serialized per threshold send, then cleared) and passes on current staging. It is independent of the size-split coverage that #39880 added for LIT-5899. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(azure_sentinel): gate the first send on events so later records provably arrive while it is in flight Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>		2026-09-08 16:55:40 -07:00
..
arize	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
azure_storage	fix(azure): restrict the storage credential chain to deployment identities (#39637)	2026-09-03 18:29:32 -07:00
bitbucket	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
cloudzero	fix(cloudzero): preserve late resource tags (#39873)	2026-09-05 12:10:05 -07:00
code_interpreter_interception	feat(sandbox): reuse e2b container across requests when metadata.session_id is set (#31688)	2026-06-30 18:58:09 -07:00
compression_interception	feat(spend): track prompt compression saved tokens in daily spend aggregates (#33810)	2026-07-18 17:47:54 -07:00
datadog	fix(azure_sentinel): split batches under the 1MB ingestion cap (#39880)	2026-09-05 17:15:36 -07:00
dotprompt	Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_dotprompt_model_swap	2026-08-26 15:13:57 -07:00
focus	fix(spend-tracking): hand plain dict rows to polars in the CloudZero and Focus exports	2026-09-03 18:46:01 -07:00
gcs_bucket	test: unwind environment writes in tests/test_litellm with monkeypatch (#37806)	2026-08-21 20:28:37 -07:00
gcs_pubsub	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
gitlab	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
helicone	test: run the 30 test files stranded in the second mirror (#37595)	2026-08-20 10:59:43 -07:00
langfuse	fix(langfuse): warn and drop invalid LANGFUSE_TRACING_ENVIRONMENT instead of failing requests (#38582)	2026-08-27 18:03:42 -07:00
levo	test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769)	2026-08-20 20:24:49 -07:00
litellm_agent	Agent Builder - support new experimental agent builder, to ensure agents pass compliance checks (#21817)	2026-02-21 15:32:47 -08:00
mavvrik_focus	chore: litellm oss staging (#31185)	2026-06-26 09:17:44 -07:00
newrelic	feat(newrelic): per-team cost and usage metrics via team callbacks (#37610)	2026-08-26 23:42:02 -07:00
open_telemetry	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
opik	Litellm oss staging 030626 (#29578)	2026-06-03 11:01:51 -07:00
otel	feat(otel v2): send a key's or team's whole trace to its own destination (#39654)	2026-09-08 16:23:21 -07:00
SlackAlerting	feat(alerting): slack alerts for per-user daily/monthly spend thresholds and spend anomaly detection (#38438)	2026-09-01 15:09:03 -07:00
vector_store_integrations	test(vector-stores): cover the hook's default proxy runtime wiring	2026-09-03 00:09:27 -07:00
websearch_interception	revert: restore search tool fallback when no router is configured	2026-09-01 11:16:23 -07:00
rubrik_test_helpers.py	Litellm oss staging 04 21 2026 2 (#26569)	2026-05-20 21:25:19 -07:00
test_agentops.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_anthropic_cache_control_hook.py	fix: skip one-shot Claude Code cache injection (#40175)	2026-09-07 18:03:43 -07:00
test_athina.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_azure_sentinel.py	test(azure_sentinel): pin batch_size as a per-request bound under concurrent events (#40320)	2026-09-08 16:55:40 -07:00
test_braintrust_logging.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_braintrust_span_name.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_custom_guardrail.py	fix(guardrails): allow framework-supported logging-only mode (#40267)	2026-09-08 12:18:09 -07:00
test_custom_guardrail_recursion.py	fix: recursive pydantic issue (#19531)	2026-01-22 19:56:41 -08:00
test_custom_prompt_management.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_deepeval.py	fix: use fastuuid helper (#14903)	2025-09-25 15:47:01 -07:00
test_galileo.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_guardrail_logging_sync.py	fix(guardrails): keep guardrail information in spend logs when the caller sends its own metadata	2026-07-24 16:20:44 -07:00
test_helicone.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_langfuse.py	fix(langfuse): warn and drop invalid LANGFUSE_TRACING_ENVIRONMENT instead of failing requests (#38582)	2026-08-27 18:03:42 -07:00
test_langfuse_otel.py	feat(langfuse): support langfuse_environment as a per-key dynamic callback param (#38264)	2026-08-26 16:56:55 -07:00
test_langsmith_init.py	fix(langsmith): keep root-run ids self-consistent so batch ingest stops rejecting header-tagged requests (#38116)	2026-08-24 19:47:12 -07:00
test_lunary.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_mlflow.py	fix(mlflow): prevent _stream_id_to_span leak and mlflow 2.x end_trace TypeError (#39049)	2026-09-08 16:18:06 -07:00
test_openmeter.py	test: unwind environment writes in tests/test_litellm with monkeypatch (#37806)	2026-08-21 20:28:37 -07:00
test_opentelemetry.py	fix(logging): stop billing and logging response reads as LLM calls (#36890)	2026-08-26 18:34:17 -07:00
test_opentelemetry_dynamic_imports.py	Make grpc dependency optional (#19447)	2026-01-20 19:03:52 -08:00
test_opik_utils.py	chore(ci): merge oss branch (#33784)	2026-07-17 23:22:13 +00:00
test_otel_guardrail_violation_spans.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_otel_team_attributes_matrix.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_prometheus_api_promql_escape.py	fix(prometheus): quote api_key for PromQL string literal in spend lookup	2026-05-01 21:38:19 +00:00
test_prometheus_budget_metric_guard.py	fix(prometheus): skip budget metric DB lookups when gauges are NoOpMetric (#32834)	2026-07-10 20:25:47 -07:00
test_prometheus_budget_metrics_db_lookups.py	perf(auth): negative-cache missing user/key lookups on the request hot path (#32368)	2026-07-08 09:59:57 +03:00
test_prometheus_budget_metrics_timeout.py	feat: litellm oss staging (#31935)	2026-07-03 09:27:31 +05:30
test_prometheus_cache_metrics.py	fix(prometheus): populate cache write token metrics for OpenAI-style usage (#34803)	2026-07-27 12:28:19 -07:00
test_prometheus_caller_identity.py	test(prometheus): cover caller-identity config failure cases (#38380)	2026-08-26 11:53:49 -07:00
test_prometheus_client_ip_user_agent.py	feat(prometheus): expose per-key and per-team rate limit allowed and used gauges (#39236)	2026-09-01 18:03:18 -07:00
test_prometheus_custom_metadata_label_counts.py	Fix Prometheus remaining metric zero values (#27348)	2026-05-06 17:22:20 -07:00
test_prometheus_end_user_cardinality.py	perf: cap Prometheus end-user metric cardinality with TTL + LRU eviction (#27272)	2026-05-06 13:35:13 -07:00
test_prometheus_invalid_key_filtering.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_prometheus_labels.py	test: rename tests that a later definition shadowed	2026-08-12 11:15:54 -07:00
test_prometheus_mcp_tool_metrics.py	feat(prometheus): expose MCP tool metadata in Prometheus metrics (#31899)	2026-07-02 10:56:35 +03:00
test_prometheus_media_generation_metrics.py	feat(prometheus): expose video duration and image count consumption metrics (#33138)	2026-07-13 18:51:13 -07:00
test_prometheus_metric_name_consistency.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_prometheus_metrics_endpoint.py	perf(prometheus): render /metrics off the event loop and coalesce concurrent scrapes (#37702)	2026-08-20 16:08:22 -07:00
test_prometheus_missing_metrics.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_prometheus_none_metadata.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_prometheus_overhead_with_guardrails.py	feat(prometheus): add litellm_total_overhead_latency_metric (SDK overhead + guardrails) (#31593)	2026-06-30 17:34:17 +08:00
test_prometheus_queue_guardrail_metrics.py	fix(prometheus): fold auth/pre-call time into litellm_request_total_latency_metric (#37958)	2026-08-22 14:25:55 -07:00
test_prometheus_rate_limit_labels.py	feat(prometheus): expose per-key and per-team rate limit allowed and used gauges (#39236)	2026-09-01 18:03:18 -07:00
test_prometheus_remaining_tokens_router_fallback.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_prometheus_requested_model_cardinality.py	fix(prometheus): pass through router-originated labels when no proxy router exists	2026-09-01 12:14:27 -07:00
test_prometheus_service_tier_label.py	feat(otel): stamp service tier attributes on inference spans (#35679)	2026-08-03 23:10:01 -07:00
test_prometheus_services.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_prometheus_spend_logs_metadata.py	fix(prometheus): expose project_alias in custom metadata labels (LIT-3741) (#31784)	2026-07-01 10:44:02 +08:00
test_prometheus_stream_label.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_prometheus_token_detail_metrics.py	fix(prometheus): populate cache write token metrics for OpenAI-style usage (#34803)	2026-07-27 12:28:19 -07:00
test_prometheus_user_team_metrics.py	fix(proxy): count only active users toward license seat limit (#31227)	2026-06-29 18:01:02 -07:00
test_prompt_manager_ssti.py	fix(security): sandbox jinja2 in gitlab/arize/bitbucket prompt managers	2026-05-02 09:14:02 +00:00
test_responses_background_cost.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
test_rubrik.py	fix(guardrails/rubrik): attribute blocked requests to the caller that made them (#35734)	2026-08-03 19:56:24 -07:00
test_s3.py	fix(s3): bound s3 object keys and download filenames for long Responses API ids (#39164)	2026-09-01 13:30:02 -07:00
test_s3_v2.py	fix(s3): bound s3 object keys and download filenames for long Responses API ids (#39164)	2026-09-01 13:30:02 -07:00
test_shadow_eval_logger.py	fix(shadow_eval): import Final for the test helper's annotation	2026-09-05 09:49:00 -07:00
test_weave_otel.py	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00