scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-16 23:41:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103
litellm/tests/test_litellm/integrations
History
yassin b9dd397746 fix(prometheus): count 401 auth failures in litellm_proxy_failed_requests_metric 
Invalid or unknown virtual keys were filtered out of the proxy failed and total
request counters entirely. Count them with hashed_api_key unset so caller-chosen
key strings cannot create unbounded label series, and normalize the request
route on the auth failure path so dynamic path ids do not leak into the route
label either.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 02:09:00 +00:00
..
arize test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
azure_storage fix(azure): restrict the storage credential chain to deployment identities (#39637) 2026-09-03 18:29:32 -07:00
bitbucket test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
cloudzero fix(cloudzero): preserve late resource tags (#39873) 2026-09-05 12:10:05 -07:00
code_interpreter_interception feat(sandbox): reuse e2b container across requests when metadata.session_id is set (#31688) 2026-06-30 18:58:09 -07:00
compression_interception feat(spend): track prompt compression saved tokens in daily spend aggregates (#33810) 2026-07-18 17:47:54 -07:00
datadog fix(datadog_llm_obs): keep tool call and result structure under redaction and emit tool output tokens (#40666) 2026-09-11 13:12:14 -07:00
dotprompt Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_dotprompt_model_swap 2026-08-26 15:13:57 -07:00
focus fix(spend-tracking): hand plain dict rows to polars in the CloudZero and Focus exports 2026-09-03 18:46:01 -07:00
gcs_bucket fix(proxy): read a bucket config's include tree off the event loop 2026-09-06 02:05:18 -07:00
gcs_pubsub test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
gitlab test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
helicone test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
langfuse fix(langfuse): warn and drop invalid LANGFUSE_TRACING_ENVIRONMENT instead of failing requests (#38582) 2026-08-27 18:03:42 -07:00
levo test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
litellm_agent Agent Builder - support new experimental agent builder, to ensure agents pass compliance checks (#21817) 2026-02-21 15:32:47 -08:00
mavvrik_focus chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
newrelic feat(newrelic): export team max and remaining budget gauges to the Metric API (#40542) 2026-09-10 17:16:15 -07:00
open_telemetry test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
opik Litellm oss staging 030626 (#29578) 2026-06-03 11:01:51 -07:00
otel Merge pull request #40562 from BerriAI/litellm_otel_openinference_message_cap 2026-09-14 11:55:37 -07:00
pointfive feat(pointfive): add batching logger callback 2026-09-10 14:02:35 +03:00
SlackAlerting fix(proxy): emit internal user budget alerts 2026-09-09 18:44:24 +08:00
vector_store_integrations fix(vector-stores): report search failures on the responses API surface too 2026-09-03 01:54:09 -07:00
websearch_interception revert: restore search tool fallback when no router is configured 2026-09-01 11:16:23 -07:00
rubrik_test_helpers.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_agentops.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_anthropic_cache_control_hook.py fix: skip one-shot Claude Code cache injection (#40175) 2026-09-07 18:03:43 -07:00
test_athina.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_azure_sentinel.py test(azure_sentinel): pin batch_size as a per-request bound under concurrent events (#40320) 2026-09-08 16:55:40 -07:00
test_braintrust_logging.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_braintrust_span_name.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_custom_guardrail.py test(guardrails): type the native lifecycle logging_only test double 2026-09-13 08:47:47 +00:00
test_custom_guardrail_recursion.py fix: recursive pydantic issue (#19531) 2026-01-22 19:56:41 -08:00
test_custom_prompt_management.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_deepeval.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_galileo.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_guardrail_logging_sync.py fix(guardrails): keep guardrail information in spend logs when the caller sends its own metadata 2026-07-24 16:20:44 -07:00
test_helicone.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_langfuse.py fix(langfuse): give each call in a session header its own trace instead of upserting one trace per session (#40177) 2026-09-09 15:56:38 -07:00
test_langfuse_otel.py feat(langfuse): support langfuse_environment as a per-key dynamic callback param (#38264) 2026-08-26 16:56:55 -07:00
test_langsmith_init.py test: assert LangSmith periodic flush by observing a batch send 2026-09-05 10:36:15 +00:00
test_lunary.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_mlflow.py fix(mlflow): prevent _stream_id_to_span leak and mlflow 2.x end_trace TypeError (#39049) 2026-09-08 16:18:06 -07:00
test_openmeter.py test: unwind environment writes in tests/test_litellm with monkeypatch (#37806) 2026-08-21 20:28:37 -07:00
test_opentelemetry.py fix(logging): stop billing and logging response reads as LLM calls (#36890) 2026-08-26 18:34:17 -07:00
test_opentelemetry_dynamic_imports.py Make grpc dependency optional (#19447) 2026-01-20 19:03:52 -08:00
test_opik_utils.py chore(ci): merge oss branch (#33784) 2026-07-17 23:22:13 +00:00
test_otel_guardrail_violation_spans.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_otel_team_attributes_matrix.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_prometheus_api_promql_escape.py fix(prometheus): quote api_key for PromQL string literal in spend lookup 2026-05-01 21:38:19 +00:00
test_prometheus_budget_metric_guard.py fix(prometheus): skip budget metric DB lookups when gauges are NoOpMetric (#32834) 2026-07-10 20:25:47 -07:00
test_prometheus_budget_metrics_db_lookups.py perf(auth): negative-cache missing user/key lookups on the request hot path (#32368) 2026-07-08 09:59:57 +03:00
test_prometheus_budget_metrics_timeout.py feat: litellm oss staging (#31935) 2026-07-03 09:27:31 +05:30
test_prometheus_cache_metrics.py fix(prometheus): populate cache write token metrics for OpenAI-style usage (#34803) 2026-07-27 12:28:19 -07:00
test_prometheus_caller_identity.py test(prometheus): cover caller-identity config failure cases (#38380) 2026-08-26 11:53:49 -07:00
test_prometheus_carried_budget_state.py perf(proxy): one MGET and one pipeline for post-call spend counters, no team/user/org refetch on the response path (#40841) 2026-09-12 16:05:50 +00:00
test_prometheus_client_ip_user_agent.py feat(prometheus): bucket latency by input sequence length (#40059) 2026-09-10 00:29:28 -07:00
test_prometheus_custom_metadata_label_counts.py Fix Prometheus remaining metric zero values (#27348) 2026-05-06 17:22:20 -07:00
test_prometheus_end_user_cardinality.py perf: cap Prometheus end-user metric cardinality with TTL + LRU eviction (#27272) 2026-05-06 13:35:13 -07:00
test_prometheus_input_sequence_length_label.py feat(prometheus): bucket latency by input sequence length (#40059) 2026-09-10 00:29:28 -07:00
test_prometheus_invalid_key_filtering.py fix(prometheus): count 401 auth failures in litellm_proxy_failed_requests_metric 2026-09-15 02:09:00 +00:00
test_prometheus_labels.py fix(prometheus): label pre-call rate limit failures with the resolved api_provider 2026-09-14 19:50:21 +00:00
test_prometheus_mcp_tool_metrics.py feat(prometheus): expose MCP tool metadata in Prometheus metrics (#31899) 2026-07-02 10:56:35 +03:00
test_prometheus_media_generation_metrics.py feat(prometheus): expose video duration and image count consumption metrics (#33138) 2026-07-13 18:51:13 -07:00
test_prometheus_metric_name_consistency.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_prometheus_metrics_endpoint.py perf(prometheus): render /metrics off the event loop and coalesce concurrent scrapes (#37702) 2026-08-20 16:08:22 -07:00
test_prometheus_missing_metrics.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_prometheus_none_metadata.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_prometheus_overhead_with_guardrails.py feat(prometheus): add litellm_total_overhead_latency_metric (SDK overhead + guardrails) (#31593) 2026-06-30 17:34:17 +08:00
test_prometheus_queue_guardrail_metrics.py fix(prometheus): fold auth/pre-call time into litellm_request_total_latency_metric (#37958) 2026-08-22 14:25:55 -07:00
test_prometheus_rate_limit_labels.py feat(prometheus): expose per-key and per-team rate limit allowed and used gauges (#39236) 2026-09-01 18:03:18 -07:00
test_prometheus_remaining_tokens_router_fallback.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_prometheus_requested_model_cardinality.py fix(prometheus): pass through router-originated labels when no proxy router exists 2026-09-01 12:14:27 -07:00
test_prometheus_service_tier_label.py feat(otel): stamp service tier attributes on inference spans (#35679) 2026-08-03 23:10:01 -07:00
test_prometheus_services.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_prometheus_spend_logs_metadata.py fix(prometheus): expose project_alias in custom metadata labels (LIT-3741) (#31784) 2026-07-01 10:44:02 +08:00
test_prometheus_stream_label.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_prometheus_token_detail_metrics.py fix(prometheus): populate cache write token metrics for OpenAI-style usage (#34803) 2026-07-27 12:28:19 -07:00
test_prometheus_user_team_metrics.py fix(proxy): count only active users toward license seat limit (#31227) 2026-06-29 18:01:02 -07:00
test_prompt_manager_ssti.py fix(security): sandbox jinja2 in gitlab/arize/bitbucket prompt managers 2026-05-02 09:14:02 +00:00
test_responses_background_cost.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_rubrik.py fix(guardrails/rubrik): attribute blocked requests to the caller that made them (#35734) 2026-08-03 19:56:24 -07:00
test_s3.py fix(s3): bound s3 object keys and download filenames for long Responses API ids (#39164) 2026-09-01 13:30:02 -07:00
test_s3_v2.py fix(s3_v2): freeze refreshable credentials before signing and retry 403 uploads with a fresh signature (#40187) 2026-09-09 23:14:16 -07:00
test_shadow_eval_logger.py fix(shadow-eval): skip hosted web search samples (#40827) 2026-09-12 02:11:48 +00:00
test_weave_otel.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00