scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-11 03:38:38 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 82
litellm/tests/unit/integrations
History
devin-ai-integration[bot] 4c6c84afb7
perf(proxy): stop prompt-cache eligibility from tokenizing the whole conversation (#44221) 
is_prompt_caching_valid_prompt ran the full Python token_counter over every message to compare against the deployment's prompt cache minimum, 500 to 1000 ms at 440k to 740k tokens on every request that reaches the prompt_caching pre-call check, Rust on or off. messages_reach_token_count does the same arithmetic as token_counter(...) >= threshold and stops at the first message that reaches the threshold. Groups with one healthy deployment skip the prefix hash and pin lookup, which cannot change the result for them

Four fixed name span events make the pre-LLM phases measurable with OTel v2: litellm.request.body_received (with body_bytes) once per body read before parsing, on the JSON, binary and form branches, body_parsed, pre_call_completed, and deployment_selected emitted once per pick inside Router.async_get_available_deployment and get_available_deployment with attempt, reason and model group, so every router surface, retry and fallback is covered. Measured locally on /v1/chat/completions, /v1/messages and /v1/responses at 440k tokens with Rust on and off against a fake upstream

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-03 09:20:28 -07:00
..
arize test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
azure_storage fix(azure_storage): keep client call ids from sharing one Data Lake file (#44099) 2026-10-01 23:23:37 -07:00
bitbucket test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
cloudzero test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
code_interpreter_interception test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
compression_interception test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
datadog test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
dotprompt test: finish the non-proxy half of tests/test_litellm (#43281) 2026-09-25 22:43:41 -07:00
focus test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
gcs_bucket test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
gcs_pubsub test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
gitlab test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
helicone test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
langfuse chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
levo test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
litellm_agent test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
mavvrik_focus test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
newrelic test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
open_telemetry feat: improve trace ingestion and trace details (#43975) 2026-10-01 13:45:33 -07:00
opik test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
otel perf(proxy): stop prompt-cache eligibility from tokenizing the whole conversation (#44221) 2026-10-03 09:20:28 -07:00
pointfive chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
SlackAlerting fix(otel): nest cache spans under their operation and name service spans by purpose (#44150) 2026-10-03 09:20:27 -07:00
vector_store_integrations feat(guardrails): scan retrieved vector store chunks with the request's pre-call guardrails (#43271) 2026-09-26 14:18:38 -07:00
websearch_interception refactor(anthropic): rename experimental_pass_through to pass_through (#43329) 2026-09-26 13:00:50 -07:00
zerobus test(zerobus): move tests into active CI selection (#43235) 2026-09-25 23:35:15 +00:00
__init__.py test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
conftest.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
rubrik_test_helpers.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_adaptive_concurrency.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
test_agentops.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_anthropic_cache_control_hook.py fix(caching): stand default cache points down when extra_body hides a direct client mark (#43341) 2026-09-26 16:44:30 -07:00
test_athina.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_azure_sentinel.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_braintrust_logging.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_braintrust_span_name.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_custom_guardrail.py feat(guardrails): honor litellm_params.timeout in every HTTP guardrail (#43134) 2026-09-30 21:18:47 -07:00
test_custom_guardrail_recursion.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_custom_prompt_management.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_deepeval.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_galileo.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_guardrail_logging_sync.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_helicone.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_langfuse.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_langfuse_otel.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_langsmith_init.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_lunary.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_mlflow.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_openmeter.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_opentelemetry.py fix(langtrace): deliver spans to app.langtrace.ai/api/trace with x-api-key (#43322) 2026-09-26 16:31:52 -07:00
test_opentelemetry_dynamic_imports.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_opik_utils.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_otel_guardrail_violation_spans.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_otel_team_attributes_matrix.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_api_promql_escape.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metric_guard.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metrics_db_lookups.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metrics_timeout.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_cache_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_caller_identity.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_carried_budget_state.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_client_ip_user_agent.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_custom_metadata_label_counts.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_deployment_state_proxy_rejects.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_end_user_cardinality.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_input_sequence_length_label.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_invalid_key_filtering.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_labels.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_mcp_tool_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_media_generation_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_metric_name_consistency.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_metrics_endpoint.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_missing_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_none_metadata.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_overhead_with_guardrails.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_queue_guardrail_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_rate_limit_labels.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_remaining_tokens_router_fallback.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_requested_model_cardinality.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_service_tier_label.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_services.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_spend_capture_rate.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_spend_logs_metadata.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_stream_label.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_token_detail_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_user_team_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_zero_cost_metric.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prompt_manager_ssti.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_responses_background_cost.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_rubrik.py feat(guardrails): honor litellm_params.timeout in every HTTP guardrail (#43134) 2026-09-30 21:18:47 -07:00
test_s3.py feat(s3_v2): add s3_partition_granularity option for hourly S3 folders (#43748) 2026-10-01 11:00:21 -07:00
test_s3_v2.py feat(s3_v2): add s3_partition_granularity option for hourly S3 folders (#43748) 2026-10-01 11:00:21 -07:00
test_shadow_eval_logger.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_weave_otel.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00