scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-08 22:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 78
litellm/tests/test_litellm/integrations
History
Exact
tin-berri 8b6ea72845
feat(shadow_eval): scope a job to model groups, ANDed with its key, team, and user targets (#39828) 
A shadow eval job could only be scoped by identity, so "this user's traffic on model X
across every key they own" was not expressible and a models field on the start body was
silently dropped. The job now carries a models list that every target is narrowed to,
matched on the requested model group with model_group_alias resolved on both sides. An
unresolvable name is a 400 at start. Empty means every model, which is what every existing
row reads as. The dashboard start form gains an "Only on models" picker and the job
headline shows the scope.
2026-09-04 20:50:46 -07:00
..
arize test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
azure_storage fix(azure): restrict the storage credential chain to deployment identities (#39637) 2026-09-03 18:29:32 -07:00
bitbucket test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
cloudzero fix(spend-tracking): hand plain dict rows to polars in the CloudZero and Focus exports 2026-09-03 18:46:01 -07:00
code_interpreter_interception feat(sandbox): reuse e2b container across requests when metadata.session_id is set (#31688) 2026-06-30 18:58:09 -07:00
compression_interception feat(spend): track prompt compression saved tokens in daily spend aggregates (#33810) 2026-07-18 17:47:54 -07:00
datadog fix(datadog_llm_obs): keep the guardrail audit record under message redaction (#39702) 2026-09-04 18:24:16 -07:00
dotprompt Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_dotprompt_model_swap 2026-08-26 15:13:57 -07:00
focus fix(spend-tracking): hand plain dict rows to polars in the CloudZero and Focus exports 2026-09-03 18:46:01 -07:00
gcs_bucket test: unwind environment writes in tests/test_litellm with monkeypatch (#37806) 2026-08-21 20:28:37 -07:00
gcs_pubsub test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
gitlab test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
helicone test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
langfuse fix(langfuse): warn and drop invalid LANGFUSE_TRACING_ENVIRONMENT instead of failing requests (#38582) 2026-08-27 18:03:42 -07:00
levo test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
litellm_agent Agent Builder - support new experimental agent builder, to ensure agents pass compliance checks (#21817) 2026-02-21 15:32:47 -08:00
mavvrik_focus chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
newrelic feat(newrelic): per-team cost and usage metrics via team callbacks (#37610) 2026-08-26 23:42:02 -07:00
open_telemetry test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
opik Litellm oss staging 030626 (#29578) 2026-06-03 11:01:51 -07:00
otel feat(otel): stamp litellm.request.route on the LLM call span (#39698) 2026-09-05 03:33:44 +00:00
SlackAlerting feat(alerting): slack alerts for per-user daily/monthly spend thresholds and spend anomaly detection (#38438) 2026-09-01 15:09:03 -07:00
vector_store_integrations test(vector-stores): cover the hook's default proxy runtime wiring 2026-09-03 00:09:27 -07:00
websearch_interception revert: restore search tool fallback when no router is configured 2026-09-01 11:16:23 -07:00
rubrik_test_helpers.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_agentops.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_anthropic_cache_control_hook.py fix(spend): keep every-deployment scope on gateway cache-injection marks 2026-09-01 17:44:29 -07:00
test_athina.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_azure_sentinel.py test: remove tests that never execute 2026-08-12 10:45:38 -07:00
test_braintrust_logging.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_braintrust_span_name.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_custom_guardrail.py Merge branch 'litellm_internal_staging' into litellm_fix_search_results_with_guardrails 2026-09-02 09:44:59 -07:00
test_custom_guardrail_recursion.py fix: recursive pydantic issue (#19531) 2026-01-22 19:56:41 -08:00
test_custom_prompt_management.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_deepeval.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_galileo.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_guardrail_logging_sync.py fix(guardrails): keep guardrail information in spend logs when the caller sends its own metadata 2026-07-24 16:20:44 -07:00
test_helicone.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_langfuse.py fix(langfuse): warn and drop invalid LANGFUSE_TRACING_ENVIRONMENT instead of failing requests (#38582) 2026-08-27 18:03:42 -07:00
test_langfuse_otel.py feat(langfuse): support langfuse_environment as a per-key dynamic callback param (#38264) 2026-08-26 16:56:55 -07:00
test_langsmith_init.py fix(langsmith): keep root-run ids self-consistent so batch ingest stops rejecting header-tagged requests (#38116) 2026-08-24 19:47:12 -07:00
test_lunary.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_mlflow.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_openmeter.py test: unwind environment writes in tests/test_litellm with monkeypatch (#37806) 2026-08-21 20:28:37 -07:00
test_opentelemetry.py fix(logging): stop billing and logging response reads as LLM calls (#36890) 2026-08-26 18:34:17 -07:00
test_opentelemetry_dynamic_imports.py Make grpc dependency optional (#19447) 2026-01-20 19:03:52 -08:00
test_opik_utils.py chore(ci): merge oss branch (#33784) 2026-07-17 23:22:13 +00:00
test_otel_guardrail_violation_spans.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_otel_team_attributes_matrix.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_prometheus_api_promql_escape.py fix(prometheus): quote api_key for PromQL string literal in spend lookup 2026-05-01 21:38:19 +00:00
test_prometheus_budget_metric_guard.py fix(prometheus): skip budget metric DB lookups when gauges are NoOpMetric (#32834) 2026-07-10 20:25:47 -07:00
test_prometheus_budget_metrics_db_lookups.py perf(auth): negative-cache missing user/key lookups on the request hot path (#32368) 2026-07-08 09:59:57 +03:00
test_prometheus_budget_metrics_timeout.py feat: litellm oss staging (#31935) 2026-07-03 09:27:31 +05:30
test_prometheus_cache_metrics.py fix(prometheus): populate cache write token metrics for OpenAI-style usage (#34803) 2026-07-27 12:28:19 -07:00
test_prometheus_caller_identity.py test(prometheus): cover caller-identity config failure cases (#38380) 2026-08-26 11:53:49 -07:00
test_prometheus_client_ip_user_agent.py feat(prometheus): expose per-key and per-team rate limit allowed and used gauges (#39236) 2026-09-01 18:03:18 -07:00
test_prometheus_custom_metadata_label_counts.py Fix Prometheus remaining metric zero values (#27348) 2026-05-06 17:22:20 -07:00
test_prometheus_end_user_cardinality.py perf: cap Prometheus end-user metric cardinality with TTL + LRU eviction (#27272) 2026-05-06 13:35:13 -07:00
test_prometheus_invalid_key_filtering.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_prometheus_labels.py test: rename tests that a later definition shadowed 2026-08-12 11:15:54 -07:00
test_prometheus_mcp_tool_metrics.py feat(prometheus): expose MCP tool metadata in Prometheus metrics (#31899) 2026-07-02 10:56:35 +03:00
test_prometheus_media_generation_metrics.py feat(prometheus): expose video duration and image count consumption metrics (#33138) 2026-07-13 18:51:13 -07:00
test_prometheus_metric_name_consistency.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_prometheus_metrics_endpoint.py perf(prometheus): render /metrics off the event loop and coalesce concurrent scrapes (#37702) 2026-08-20 16:08:22 -07:00
test_prometheus_missing_metrics.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_prometheus_none_metadata.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_prometheus_overhead_with_guardrails.py feat(prometheus): add litellm_total_overhead_latency_metric (SDK overhead + guardrails) (#31593) 2026-06-30 17:34:17 +08:00
test_prometheus_queue_guardrail_metrics.py fix(prometheus): fold auth/pre-call time into litellm_request_total_latency_metric (#37958) 2026-08-22 14:25:55 -07:00
test_prometheus_rate_limit_labels.py feat(prometheus): expose per-key and per-team rate limit allowed and used gauges (#39236) 2026-09-01 18:03:18 -07:00
test_prometheus_remaining_tokens_router_fallback.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_prometheus_requested_model_cardinality.py fix(prometheus): pass through router-originated labels when no proxy router exists 2026-09-01 12:14:27 -07:00
test_prometheus_service_tier_label.py feat(otel): stamp service tier attributes on inference spans (#35679) 2026-08-03 23:10:01 -07:00
test_prometheus_services.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_prometheus_spend_logs_metadata.py fix(prometheus): expose project_alias in custom metadata labels (LIT-3741) (#31784) 2026-07-01 10:44:02 +08:00
test_prometheus_stream_label.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_prometheus_token_detail_metrics.py fix(prometheus): populate cache write token metrics for OpenAI-style usage (#34803) 2026-07-27 12:28:19 -07:00
test_prometheus_user_team_metrics.py fix(proxy): count only active users toward license seat limit (#31227) 2026-06-29 18:01:02 -07:00
test_prompt_manager_ssti.py fix(security): sandbox jinja2 in gitlab/arize/bitbucket prompt managers 2026-05-02 09:14:02 +00:00
test_responses_background_cost.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_rubrik.py fix(guardrails/rubrik): attribute blocked requests to the caller that made them (#35734) 2026-08-03 19:56:24 -07:00
test_s3.py fix(s3): bound s3 object keys and download filenames for long Responses API ids (#39164) 2026-09-01 13:30:02 -07:00
test_s3_v2.py fix(s3): bound s3 object keys and download filenames for long Responses API ids (#39164) 2026-09-01 13:30:02 -07:00
test_shadow_eval_logger.py feat(shadow_eval): scope a job to model groups, ANDed with its key, team, and user targets (#39828) 2026-09-04 20:50:46 -07:00
test_weave_otel.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00