scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-15 09:12:18 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 311
litellm/tests/unit/integrations
History
Exact
paveln83 3ad164382b
fix(responses): send cache_control as a prompt_cache_breakpoint on the chat-to-Responses bridge (#51783) 
* fix(responses bridge): carry cache_control as a prompt_cache_breakpoint to deployments that read them

A chat request with Anthropic-style cache_control on a system or user part got its mark renamed to
prompt_cache_breakpoint on the plain chat path, but the chat-to-Responses bridge (taken once a
function tool and reasoning are on, and always for bedrock_mantle GPT) dropped it, so explicit-mode
caching never wrote. The bridge now renames the marks before converting the messages when the
served deployment reads breakpoints, using the same gate the Responses config uses for native
/v1/responses input. Deployments that do not read breakpoints keep today's behavior.

The rename also leaves unmarked content parts untouched instead of validating them, so a part with
non-string keys no longer raises on either path.

* fix(responses): keep leading system messages in client order once one goes to input

A marked first system string became a list and moved to input while the
next unmarked string stayed in instructions, so the model read them in
reverse. The leading run that joins instructions now stops at the first
system message whose content is not a string

* fix(responses): leave Bedrock Mantle and unmarked system order as they were on the bridge

Bedrock Mantle already caches the growing prefix on its own, so turning
cache_control into breakpoints there added write charges. The client-order
rule for leading system messages now applies only when a mark turns a system
string into a list, so unmarked chats send the same request as before.

* test(responses): cover a marked user turn in the unmarked system split test

A cache_control mark on the user message must not switch unmarked leading
system messages to the client-order rule. The split test now runs with the
user turn plain and marked, so dropping the role check from the system
string test fails it.

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-11-14 05:00:49 -05:00
..
arize refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
azure_storage fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493) 2026-11-02 14:13:17 -08:00
bitbucket refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
clickhouse fix(router): count a Responses WebSocket turn's tokens before the next turn is admitted (#50730) 2026-11-10 04:13:51 +00:00
cloudzero test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
code_interpreter_interception fix(tool_choice): accept the Responses API's flat allowed_tools layout on chat completions (#50467) 2026-11-10 13:04:06 -08:00
compression_interception fix(tool_choice): accept the Responses API's flat allowed_tools layout on chat completions (#50467) 2026-11-10 13:04:06 -08:00
custom_logger_v2 feat(custom-logger-v2): add the contract for rust callbacks and bind the proxy request scope (#51526) 2026-11-14 02:33:27 +00:00
datadog fix(anthropic)!: do not surface server_tool_use as client tool_calls (#48385) 2026-11-06 03:54:34 -08:00
dotprompt refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
email_alerting feat(alerts): add management controls and opt-in daily summaries (#51418) 2026-11-12 10:06:07 -08:00
focus refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
gcs_bucket fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493) 2026-11-02 14:13:17 -08:00
gcs_pubsub fix(vertex_ai): reuse one Google token across RAG Engine searches and cold bursts (#51239) 2026-11-14 03:12:09 -05:00
generic_api fix(proxy): show committed spend in the remaining budget gauges (#49199) 2026-11-09 20:05:37 +00:00
gitlab refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
helicone test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
langfuse fix(cost): preserve Bedrock cross-region pricing for Messages (#51479) 2026-11-12 15:54:56 -08:00
levo test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
litellm_agent test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
mavvrik_focus refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
newrelic fix(anthropic)!: do not surface server_tool_use as client tool_calls (#48385) 2026-11-06 03:54:34 -08:00
open_telemetry fix(proxy): trace model list requests with the caller's traceparent (#49141) 2026-10-31 01:22:45 -07:00
opik refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
otel refactor(redis): delete the Python RedisBatch now that every request op is native (#51383) 2026-11-13 03:22:28 +00:00
pointfive chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
SlackAlerting feat(alerts): add management controls and opt-in daily summaries (#51418) 2026-11-12 10:06:07 -08:00
ternary feat(focus): add ternary destination for FOCUS export (#46106) 2026-10-12 22:52:40 -07:00
vantage chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
vector_store_integrations feat(ui): show customer organization usage and budget utilization (#51483) 2026-11-12 17:25:41 -08:00
websearch_interception fix(tool_choice): accept the Responses API's flat allowed_tools layout on chat completions (#50467) 2026-11-10 13:04:06 -08:00
zerobus test(zerobus): move tests into active CI selection (#43235) 2026-09-25 23:35:15 +00:00
__init__.py test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
conftest.py fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493) 2026-11-02 14:13:17 -08:00
rubrik_test_helpers.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_adaptive_concurrency.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
test_agentops.py fix(caching): key cached async clients by a per-loop token instead of id(loop) (#48877) 2026-10-29 13:14:48 -07:00
test_anthropic_cache_control_hook.py fix(responses): send cache_control as a prompt_cache_breakpoint on the chat-to-Responses bridge (#51783) 2026-11-14 05:00:49 -05:00
test_argilla.py chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
test_athina.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_azure_sentinel.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_braintrust_logging.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_braintrust_span_name.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_custom_callback_input.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_custom_guardrail.py fix(responses): run the guardrails a Responses WebSocket frame selects (#48725) 2026-10-30 14:12:51 -07:00
test_custom_guardrail_post_call_once.py feat(custom-logger-v2): add types, dispatcher and route callback call sites (step 0) (#51214) 2026-11-13 22:08:05 +00:00
test_custom_guardrail_recursion.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_custom_logger.py feat(custom-logger-v2): add types, dispatcher and route callback call sites (step 0) (#51214) 2026-11-13 22:08:05 +00:00
test_custom_prompt_management.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_customer_budget_webhook.py fix(proxy): alert on customer org, customer team, and customer member budgets (#51579) 2026-11-13 09:16:33 -08:00
test_deepeval.py fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493) 2026-11-02 14:13:17 -08:00
test_dynamodb.py fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493) 2026-11-02 14:13:17 -08:00
test_email_alerting.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_galileo.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_guardrail_logging_sync.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_helicone.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_humanloop.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_lago.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_langfuse.py fix(logging): mark mirrored silent_model calls with is_silent_experiment in spend logs (#48888) 2026-10-29 05:12:50 -07:00
test_langfuse_otel.py fix(langfuse): reserve span capacity for failure usage attributes (#51538) 2026-11-12 21:14:14 -08:00
test_langsmith.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_langsmith_init.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_langtrace.py fix(otel): serialize pydantic Message objects in langfuse_otel, weave_otel and langtrace (#46845) 2026-11-10 14:50:39 -08:00
test_lunary.py fix(anthropic)!: do not surface server_tool_use as client tool_calls (#48385) 2026-11-06 03:54:34 -08:00
test_mlflow.py test(unit): wait on the work itself instead of fixed sleeps (#46261) 2026-10-13 12:41:06 -07:00
test_openmeter.py fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493) 2026-11-02 14:13:17 -08:00
test_opentelemetry.py fix(otel): distinguish missing usage from prompt estimates on chat failures (#51443) 2026-11-12 13:48:07 -08:00
test_opentelemetry_dynamic_imports.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_opentelemetry_request_spans.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_opik_utils.py refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
test_otel_guardrail_violation_spans.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_otel_team_attributes_matrix.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_performance_alert_notifications.py feat(alerts): add management controls and opt-in daily summaries (#51418) 2026-11-12 10:06:07 -08:00
test_posthog.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_prometheus_api_promql_escape.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metric_guard.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metrics_db_lookups.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metrics_timeout.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_cache_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_caller_identity.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_carried_budget_state.py refactor(redis): delete the Python RedisBatch now that every request op is native (#51383) 2026-11-13 03:22:28 +00:00
test_prometheus_client_ip_user_agent.py feat(prometheus): expose per-project per-model rate limit allowed and used gauges (#43561) 2026-10-08 12:03:29 -05:00
test_prometheus_custom_metadata_label_counts.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_deployment_state_proxy_rejects.py fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
test_prometheus_end_user_cardinality.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_input_sequence_length_label.py feat(ui): add auto-router usage and savings table (#45152) 2026-10-07 15:04:07 -07:00
test_prometheus_invalid_key_filtering.py fix(azure)!: answer 401 when Entra rejects the deployment's credential (#49863) 2026-11-04 20:44:55 +00:00
test_prometheus_labels.py feat(dynamic_rate_limiter_v3): prometheus share, borrowed and throttle metrics per team and model (#51417) 2026-11-12 15:28:51 -08:00
test_prometheus_mcp_tool_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_media_generation_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_metric_name_consistency.py chore: remove cookbook and preserve live assets (#50944) 2026-11-11 15:38:27 -08:00
test_prometheus_metrics_endpoint.py test(unit): wait on the work itself instead of fixed sleeps (#46261) 2026-10-13 12:41:06 -07:00
test_prometheus_missing_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_none_metadata.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_overhead_with_guardrails.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_queue_guardrail_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_rate_limit_labels.py fix(prometheus): skip custom metadata labels a metric already carries (#46267) 2026-10-13 18:20:52 -07:00
test_prometheus_remaining_tokens_router_fallback.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_prometheus_requested_model_cardinality.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_series_cardinality.py feat(prometheus): cap series per metric for every labeled metric (#44420) 2026-10-06 01:28:44 +00:00
test_prometheus_service_tier_label.py fix(bedrock): answer 400 for service tiers Bedrock cannot serve, forward reserved (#50516) 2026-11-14 01:33:57 +00:00
test_prometheus_services.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_prometheus_spend_capture_rate.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_spend_logs_metadata.py fix(logging): mark mirrored silent_model calls with is_silent_experiment in spend logs (#48888) 2026-10-29 05:12:50 -07:00
test_prometheus_stream_label.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_token_detail_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_user_team_metrics.py fix(proxy): show committed spend in the remaining budget gauges (#49199) 2026-11-09 20:05:37 +00:00
test_prometheus_zero_cost_metric.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prompt_layer.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_prompt_manager_ssti.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_responses_background_cost.py fix(responses): send the billed background response cost to the creating key's and team's loggers (#47506) 2026-10-20 23:22:21 +00:00
test_rubrik.py fix(guardrails/rubrik): moderate text-completion prompts and forward rac_agent_id (#49912) 2026-11-10 16:30:41 -08:00
test_s3.py feat(s3_v2): add s3_partition_granularity option for hourly S3 folders (#43748) 2026-10-01 11:00:21 -07:00
test_s3_v2.py fix(s3): bound object key filenames (#40825) 2026-10-15 12:59:35 -07:00
test_shadow_eval_logger.py fix(anthropic)!: do not surface server_tool_use as client tool_calls (#48385) 2026-11-06 03:54:34 -08:00
test_sqs.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_wandb.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_weave_otel.py fix(otel): serialize pydantic Message objects in langfuse_otel, weave_otel and langtrace (#46845) 2026-11-10 14:50:39 -08:00