mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-10 08:28:25 +00:00
71babd931b
* fix(transformation): merge consecutive system and developer messages for non-OpenAI providers and Responses API
* fix(base_utils): keep later cache_control and billing metadata boundaries when merging system messages
* perf(base_utils): fold consecutive system messages in a single linear pass
The tuple accumulator passed to reduce copied the whole prefix on every step,
making map_developer_role_to_system_role O(n^2) in the number of messages.
Message count is not bounded by default, so a caller could tie up a proxy
worker with thousands of small messages before any provider call. Use a list
and merge into its last element in place instead.
* fix(base_utils): hoist later developer messages into the leading system message for non-OpenAI providers
A developer message that arrives after the first user turn (Codex CLI's
per-turn permissions item, a re-injected instruction after compaction, a
mid-conversation policy update) was translated to a system message in
place, and system-first chat templates such as Fireworks' qwen3.8 reject
any system message that is not at the beginning with 400. The fold that
merges consecutive system messages never saw it because a user turn sat
in between.
Move every developer message that follows the leading system block to
the end of that block before the fold, so it merges into the single
leading system message; client-authored mid-conversation system messages
stay where the client put them.
* fix(openai): hoist later developer messages into one leading system message on OpenAI-compatible backends
Developer role translation moves onto the provider config. OpenAI's own endpoint,
Azure OpenAI, and the o-series keep their current behavior. Every other config
inheriting OpenAIGPTConfig (Fireworks, DeepSeek, Together, vLLM, Groq, openai/
with a custom api_base) hoists developer messages that arrive after the leading
system block into that block and folds the block into a single system message,
which is what system-first chat templates require. Providers with their own
message format are untouched.
Messages carrying cache_control or Anthropic billing metadata fold as their own
text blocks, so breakpoints and provider-side stripping survive the merge. The
Responses bridge goes back to prepending instructions as a separate system
message and lets the hoist fold it.
* refactor(base_utils): fold each run of system messages in one pass
The pairwise fold recopied the accumulated system text or block list on every
merge, so a request carrying many consecutive system or developer messages cost
quadratic time in preprocessing. Group consecutive system messages once and
build the merged message from the whole run: plain strings join with a blank
line, and any cached, billing, or block content keeps one text block per
message. The name of the last named message in the run survives, as before.
* fix(base_utils): keep a closing developer message in place and hoist for OpenAIConfig backends
A developer message that closes the conversation stays where the client put it, translated
to system in place, so the request never ends on the assistant's turn. OpenAIConfig and its
subclasses (Azure AI Foundry, GitHub Copilot, ChatGPT, RAGFlow) now share the OpenAI-compatible
hoist, litellm_proxy passes developer messages through to the downstream proxy, and the Anthropic
billing header literals use the shared constant
* fix(base_utils): keep a closing developer message in place only after an assistant turn
A developer message that closes the conversation after a user or tool message hoists like
any other, so the request shape the ticket reports keeps working on system-first templates;
only a closing run that follows an assistant turn stays in place
* refactor(openai): reuse is_openai_backed_api_base for the hosted endpoint check
* test(base_utils): cover an empty system message inside a block-content run
Codecov flagged the empty-string branch of _text_blocks as the one line the
PR left uncovered. The existing empty-string test only hits the plain-text
join; this one forces the block path by merging an empty system message with
a developer message that carries list content.
* fix(openai): hoist developer messages for litellm_proxy and harden the system fold
- drop the LiteLLMProxyChatConfig passthrough so litellm_proxy/ hoists later developer messages like every other OpenAI-compatible endpoint, whichever backend or LiteLLM version sits downstream
- a system message with null or missing content inside a folded run is dropped instead of raising KeyError('content')
- a message-level cache_control on a list-content member lands on its last block so the breakpoint survives the fold
* fix(litellm_proxy): keep the AllMessageValues import the transform signatures still name
* fix(fireworks_ai): hoist developer items after the first turn on the native Responses path
The native Fireworks Responses path landed on staging mapping developer input items to
system in place, so a developer item after the first input item still hit the same
"System message must be at the beginning" 400 on system-first chat templates. The
ordering rules now live in one shared helper over item roles, and the native path
reorders through it before mapping, keeping a closing developer item after an assistant
turn in place exactly like the chat path does
* fix(transformation): carry the developer-message hoist onto main 963333d654
Resolves the main merge for this branch: the tests move to tests/unit, every translate_developer_role_to_system_role override takes and returns a Sequence, Azure OpenAI keeps the in-place mapping, and the system content reads tolerate a missing key
* fix(transformation): join a lone plain text block as text when hoisting developer messages
The Responses bridge turns every developer item into a one-block list, so the hoisted run merged the
instructions string and that block as a block list, and Databricks could no longer collapse it into the
single system string it sends today. A lone text block with no extra keys now joins the run as text,
while multi-block, cached, and billing-header content keep the block path
* fix(transformation): hoist later developer messages only for models that reject mid-conversation system messages
Runs of consecutive system messages still fold into one, which keeps the cached prefix stable. A developer message after the first turn moves into the leading system message only when the model declares supports_mid_conversation_system false, or when it is a Fireworks model with no declaration, matching the native Fireworks Responses route. Every other backend keeps it in place as before, so their prompt cache prefix is unchanged. Malformed system content passes through unmerged so the provider returns its own 400.
* fix(responses): fold later developer items into instructions on hosted_vllm for system-first models
Moves the Responses developer fold shared by Fireworks into one module and applies it to hosted_vllm native Responses when the model declares supports_mid_conversation_system false. Renames the run-merging helper so it no longer shares a name with the Databricks one, and returns tuples from the block helpers.
* fix(llms): place developer messages only for models declared system-first
Models without a supports_mid_conversation_system declaration now keep the
in-place developer-to-system translation they had before, so their cached
prompt prefix does not change. Hoisting and folding run only when the
deployment's model_info or the cost map declares the flag false, with the
deployment value read first.
A developer message that closes the conversation after an assistant turn is
sent as a user message on those models instead of being hoisted, merged list
content keeps a blank-line separator before each later part, and the cost map
declares the flag false for fireworks_ai qwen3p8-max
* fix(proxy): keep a closing developer message's cache mark when it is sent as a user turn
* test: type the developer-hoist tests and fake HTTP with respx
* fix: count non-string roles as turns, keep the Fireworks fold for twins, and copy closing developer content
A message whose role is a list or dict now counts as a conversation turn instead of raising a 500.
Fireworks native Responses skips the developer-item fold only when the deployment's own model_info
declares supports_mid_conversation_system true, since the router registers every deployment's flag
under its shared backend key. A hoisted developer message keeps only role, content, name, and
cache_control, a closing developer message's content is deep-copied so the caller's list is left
alone, and a hosted_vllm developer item with no content is forwarded unchanged.
The Fireworks in-place test now declares true on the deployment itself, because a cost-map true can
no longer be told apart from another deployment's registration.
---------
Co-authored-by: sowmy narayanan <sowmysuresh86@gmail.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
494 lines
24 KiB
Python
494 lines
24 KiB
Python
from typing import Final
|
|
|
|
from litellm.llms.base_llm.base_utils import (
|
|
hoist_developer_messages_into_leading_system_message,
|
|
merge_system_message_runs,
|
|
)
|
|
|
|
|
|
class TestMergeSystemMessageRuns:
|
|
def test_later_developer_message_stays_in_place_while_the_leading_run_folds(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "You are Codex"},
|
|
{"role": "developer", "content": "Sandbox: read-only."},
|
|
{"role": "user", "content": "Hi there"},
|
|
{"role": "developer", "content": "Answer with exactly one word."},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
assert list(merge_system_message_runs(messages)) == [
|
|
{"role": "system", "content": "You are Codex\n\nSandbox: read-only."},
|
|
{"role": "user", "content": "Hi there"},
|
|
{"role": "system", "content": "Answer with exactly one word."},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
|
|
def test_run_with_content_that_is_neither_text_nor_blocks_is_forwarded_unmerged(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "You are Codex"},
|
|
{"role": "developer", "content": {"text": "not a content list"}},
|
|
{"role": "user", "content": "Hi there"},
|
|
]
|
|
assert list(merge_system_message_runs(messages)) == [
|
|
{"role": "system", "content": "You are Codex"},
|
|
{"role": "system", "content": {"text": "not a content list"}},
|
|
{"role": "user", "content": "Hi there"},
|
|
]
|
|
|
|
def test_hoisted_developer_message_with_content_that_is_neither_text_nor_blocks_is_forwarded_unmerged(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "You are Codex"},
|
|
{"role": "user", "content": "Hi there"},
|
|
{"role": "developer", "content": 42},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "You are Codex"},
|
|
{"role": "system", "content": 42},
|
|
{"role": "user", "content": "Hi there"},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
|
|
|
|
class TestHoistDeveloperMessagesIntoLeadingSystemMessage:
|
|
def test_single_developer_message(self) -> None:
|
|
messages: Final = [
|
|
{"role": "developer", "content": "You are a helpful assistant."},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "You are a helpful assistant."},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
|
|
def test_consecutive_developer_messages_merge(self) -> None:
|
|
messages: Final = [
|
|
{"role": "developer", "content": "Rule 1: Be concise."},
|
|
{"role": "developer", "content": "Rule 2: Respond in Markdown."},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "Rule 1: Be concise.\n\nRule 2: Respond in Markdown."},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
|
|
def test_system_followed_by_developer_merges(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "Base instructions."},
|
|
{"role": "developer", "content": "Developer override."},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "Base instructions.\n\nDeveloper override."},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
|
|
def test_developer_followed_by_system_merges(self) -> None:
|
|
messages: Final = [
|
|
{"role": "developer", "content": "Developer instruction."},
|
|
{"role": "system", "content": "System instruction."},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "Developer instruction.\n\nSystem instruction."},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
|
|
def test_consecutive_native_system_messages_merge(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "System part 1."},
|
|
{"role": "system", "content": "System part 2."},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "System part 1.\n\nSystem part 2."},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
|
|
def test_empty_string_side_does_not_pad(self) -> None:
|
|
assert list(
|
|
hoist_developer_messages_into_leading_system_message(
|
|
[{"role": "system", "content": ""}, {"role": "developer", "content": "Rules"}]
|
|
)
|
|
) == [{"role": "system", "content": "Rules"}]
|
|
assert list(
|
|
hoist_developer_messages_into_leading_system_message(
|
|
[{"role": "system", "content": "Rules"}, {"role": "developer", "content": ""}]
|
|
)
|
|
) == [{"role": "system", "content": "Rules"}]
|
|
|
|
def test_empty_string_message_adds_no_block_when_the_run_merges_as_blocks(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": ""},
|
|
{"role": "developer", "content": [{"type": "text", "text": "Rule 1"}, {"type": "text", "text": "Rule 2"}]},
|
|
{"role": "user", "content": "Hi"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": [{"type": "text", "text": "Rule 1"}, {"type": "text", "text": "Rule 2"}]},
|
|
{"role": "user", "content": "Hi"},
|
|
]
|
|
|
|
def test_two_cached_string_messages_keep_one_breakpoint_each(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "A", "cache_control": {"type": "ephemeral", "ttl": "5m"}},
|
|
{"role": "system", "content": "B", "cache_control": {"type": "ephemeral", "ttl": "1h"}},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{
|
|
"role": "system",
|
|
"content": [
|
|
{"type": "text", "text": "A", "cache_control": {"type": "ephemeral", "ttl": "5m"}},
|
|
{"type": "text", "text": "\n\nB", "cache_control": {"type": "ephemeral", "ttl": "1h"}},
|
|
],
|
|
},
|
|
]
|
|
|
|
def test_cache_control_supplied_only_by_the_later_message_marks_only_its_block(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "A"},
|
|
{"role": "developer", "content": "B", "cache_control": {"type": "ephemeral"}},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{
|
|
"role": "system",
|
|
"content": [
|
|
{"type": "text", "text": "A"},
|
|
{"type": "text", "text": "\n\nB", "cache_control": {"type": "ephemeral"}},
|
|
],
|
|
},
|
|
]
|
|
|
|
def test_cache_control_of_the_first_message_stays_on_its_own_block(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "Cached prompt", "cache_control": {"type": "ephemeral"}},
|
|
{"role": "developer", "content": "Additional instructions"},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{
|
|
"role": "system",
|
|
"content": [
|
|
{"type": "text", "text": "Cached prompt", "cache_control": {"type": "ephemeral"}},
|
|
{"type": "text", "text": "\n\nAdditional instructions"},
|
|
],
|
|
},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
|
|
def test_cached_string_message_merged_with_block_content_keeps_its_breakpoint(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "Cached prompt", "cache_control": {"type": "ephemeral"}},
|
|
{"role": "user", "content": "Hi there"},
|
|
{"role": "developer", "content": [{"type": "text", "text": "Answer with exactly one word."}]},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{
|
|
"role": "system",
|
|
"content": [
|
|
{"type": "text", "text": "Cached prompt", "cache_control": {"type": "ephemeral"}},
|
|
{"type": "text", "text": "\n\nAnswer with exactly one word."},
|
|
],
|
|
},
|
|
{"role": "user", "content": "Hi there"},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
|
|
def test_plain_messages_ahead_of_a_cached_one_keep_their_own_blocks(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "Base"},
|
|
{"role": "developer", "content": "Rule"},
|
|
{"role": "developer", "content": "Cached", "cache_control": {"type": "ephemeral"}},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{
|
|
"role": "system",
|
|
"content": [
|
|
{"type": "text", "text": "Base"},
|
|
{"type": "text", "text": "\n\nRule"},
|
|
{"type": "text", "text": "\n\nCached", "cache_control": {"type": "ephemeral"}},
|
|
],
|
|
},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
|
|
def test_name_of_the_last_named_message_survives_a_long_run(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "Base", "name": "first"},
|
|
*({"role": "developer", "content": f"Rule {index}"} for index in range(2000)),
|
|
{"role": "developer", "content": "Last", "name": "last"},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
merged: Final = list(hoist_developer_messages_into_leading_system_message(messages))
|
|
assert len(merged) == 2
|
|
assert merged[0]["name"] == "last"
|
|
assert merged[0]["content"] == "\n\n".join(["Base", *(f"Rule {index}" for index in range(2000)), "Last"])
|
|
|
|
def test_billing_metadata_keeps_its_own_block_when_merged(self) -> None:
|
|
billing: Final = {"role": "system", "content": "x-anthropic-billing-header: cc_version=1.0;"}
|
|
advisory: Final = {"role": "system", "content": "Guardrail advisory: request was flagged"}
|
|
assert list(hoist_developer_messages_into_leading_system_message([billing, advisory])) == [
|
|
{
|
|
"role": "system",
|
|
"content": [
|
|
{"type": "text", "text": "x-anthropic-billing-header: cc_version=1.0;"},
|
|
{"type": "text", "text": "\n\nGuardrail advisory: request was flagged"},
|
|
],
|
|
}
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message([advisory, billing])) == [
|
|
{
|
|
"role": "system",
|
|
"content": [
|
|
{"type": "text", "text": "Guardrail advisory: request was flagged"},
|
|
{"type": "text", "text": "x-anthropic-billing-header: cc_version=1.0;"},
|
|
],
|
|
}
|
|
]
|
|
|
|
def test_developer_message_after_a_user_turn_is_hoisted_into_the_leading_system_message(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "System turn 1"},
|
|
{"role": "user", "content": "User turn 1"},
|
|
{"role": "developer", "content": "Developer turn 2"},
|
|
{"role": "user", "content": "User turn 2"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "System turn 1\n\nDeveloper turn 2"},
|
|
{"role": "user", "content": "User turn 1"},
|
|
{"role": "user", "content": "User turn 2"},
|
|
]
|
|
|
|
def test_hoisted_developer_message_keeps_only_the_fields_a_system_message_has(self) -> None:
|
|
messages: Final = [
|
|
{"role": "user", "content": "Hi there"},
|
|
{"role": "developer", "content": "Answer with exactly one word.", "name": "codex", "extra_key": "x"},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "Answer with exactly one word.", "name": "codex"},
|
|
{"role": "user", "content": "Hi there"},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
|
|
def test_developer_message_in_second_position_without_a_leading_system_message_is_hoisted(self) -> None:
|
|
messages: Final = [
|
|
{"role": "user", "content": "Hi there"},
|
|
{"role": "developer", "content": "Answer with exactly one word."},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "Answer with exactly one word."},
|
|
{"role": "user", "content": "Hi there"},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
|
|
def test_every_later_developer_message_is_hoisted_in_order_across_assistant_and_tool_turns(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "Base"},
|
|
{"role": "user", "content": "Turn 1"},
|
|
{"role": "assistant", "content": "Reply 1"},
|
|
{"role": "developer", "content": "Update A"},
|
|
{"role": "user", "content": "Turn 2"},
|
|
{"role": "developer", "content": "Update B"},
|
|
{"role": "user", "content": "Turn 3"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "Base\n\nUpdate A\n\nUpdate B"},
|
|
{"role": "user", "content": "Turn 1"},
|
|
{"role": "assistant", "content": "Reply 1"},
|
|
{"role": "user", "content": "Turn 2"},
|
|
{"role": "user", "content": "Turn 3"},
|
|
]
|
|
|
|
def test_developer_message_that_closes_the_conversation_after_an_assistant_turn_becomes_a_user_turn(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "You are terse."},
|
|
{"role": "user", "content": "Hi there"},
|
|
{"role": "assistant", "content": "Hello! How can I help?"},
|
|
{"role": "developer", "content": "Reply with the single word PONG and nothing else."},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "You are terse."},
|
|
{"role": "user", "content": "Hi there"},
|
|
{"role": "assistant", "content": "Hello! How can I help?"},
|
|
{"role": "user", "content": "Reply with the single word PONG and nothing else."},
|
|
]
|
|
|
|
def test_closing_developer_message_keeps_its_cache_mark_as_a_user_turn(self) -> None:
|
|
messages: Final = [
|
|
{"role": "user", "content": "Hi there"},
|
|
{"role": "assistant", "content": "Hello! How can I help?"},
|
|
{"role": "developer", "content": "Reply with PONG.", "cache_control": {"type": "ephemeral"}},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages))[-1] == {
|
|
"role": "user",
|
|
"content": "Reply with PONG.",
|
|
"cache_control": {"type": "ephemeral"},
|
|
}
|
|
|
|
def test_only_the_closing_developer_run_after_an_assistant_turn_becomes_user_turns(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "Base"},
|
|
{"role": "user", "content": "Turn 1"},
|
|
{"role": "developer", "content": "Update A"},
|
|
{"role": "user", "content": "Turn 2"},
|
|
{"role": "assistant", "content": "Reply 2"},
|
|
{"role": "developer", "content": "Closing B"},
|
|
{"role": "developer", "content": "Closing C"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "Base\n\nUpdate A"},
|
|
{"role": "user", "content": "Turn 1"},
|
|
{"role": "user", "content": "Turn 2"},
|
|
{"role": "assistant", "content": "Reply 2"},
|
|
{"role": "user", "content": "Closing B"},
|
|
{"role": "user", "content": "Closing C"},
|
|
]
|
|
|
|
def test_developer_message_that_closes_the_conversation_after_a_user_turn_is_hoisted(self) -> None:
|
|
messages: Final = [
|
|
{"role": "user", "content": "Hi there"},
|
|
{"role": "developer", "content": "Reply with exactly one word: the capital of France"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "Reply with exactly one word: the capital of France"},
|
|
{"role": "user", "content": "Hi there"},
|
|
]
|
|
|
|
def test_developer_message_that_closes_the_conversation_after_a_tool_result_is_hoisted(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "Base"},
|
|
{"role": "user", "content": "Look it up"},
|
|
{
|
|
"role": "assistant",
|
|
"content": None,
|
|
"tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "lookup", "arguments": "{}"}}],
|
|
},
|
|
{"role": "tool", "tool_call_id": "call_1", "content": "Paris"},
|
|
{"role": "developer", "content": "Answer with exactly one word."},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "Base\n\nAnswer with exactly one word."},
|
|
{"role": "user", "content": "Look it up"},
|
|
{
|
|
"role": "assistant",
|
|
"content": None,
|
|
"tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "lookup", "arguments": "{}"}}],
|
|
},
|
|
{"role": "tool", "tool_call_id": "call_1", "content": "Paris"},
|
|
]
|
|
|
|
def test_hoisted_single_text_block_developer_message_joins_string_instructions_as_text(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "You are Codex"},
|
|
{"role": "user", "content": [{"type": "text", "text": "# AGENTS.md"}]},
|
|
{"role": "developer", "content": [{"type": "text", "text": "<permissions instructions>"}]},
|
|
{"role": "user", "content": [{"type": "text", "text": "What is the capital of France?"}]},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "You are Codex\n\n<permissions instructions>"},
|
|
{"role": "user", "content": [{"type": "text", "text": "# AGENTS.md"}]},
|
|
{"role": "user", "content": [{"type": "text", "text": "What is the capital of France?"}]},
|
|
]
|
|
|
|
def test_multi_block_developer_message_keeps_its_blocks_after_string_instructions(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "You are Codex"},
|
|
{"role": "user", "content": "Hi"},
|
|
{"role": "developer", "content": [{"type": "text", "text": "Rule 1"}, {"type": "text", "text": "Rule 2"}]},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{
|
|
"role": "system",
|
|
"content": [
|
|
{"type": "text", "text": "You are Codex"},
|
|
{"type": "text", "text": "\n\nRule 1"},
|
|
{"type": "text", "text": "Rule 2"},
|
|
],
|
|
},
|
|
{"role": "user", "content": "Hi"},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
|
|
def test_system_message_without_content_folds_into_the_run_instead_of_raising(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "Rules"},
|
|
{"role": "system"},
|
|
{"role": "system", "content": None},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{"role": "system", "content": "Rules"},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
|
|
def test_message_level_cache_control_of_a_block_content_member_lands_on_its_last_block(self) -> None:
|
|
messages: Final = [
|
|
{
|
|
"role": "system",
|
|
"content": [{"type": "text", "text": "Rule 1"}, {"type": "text", "text": "Rule 2"}],
|
|
"cache_control": {"type": "ephemeral"},
|
|
},
|
|
{"role": "developer", "content": "Answer with exactly one word."},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{
|
|
"role": "system",
|
|
"content": [
|
|
{"type": "text", "text": "Rule 1"},
|
|
{"type": "text", "text": "Rule 2", "cache_control": {"type": "ephemeral"}},
|
|
{"type": "text", "text": "\n\nAnswer with exactly one word."},
|
|
],
|
|
},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
|
|
def test_message_level_cache_control_does_not_override_a_block_that_already_carries_one(self) -> None:
|
|
messages: Final = [
|
|
{
|
|
"role": "system",
|
|
"content": [{"type": "text", "text": "Rule 1", "cache_control": {"type": "ephemeral", "ttl": "1h"}}],
|
|
"cache_control": {"type": "ephemeral"},
|
|
},
|
|
{"role": "developer", "content": "Answer with exactly one word."},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{
|
|
"role": "system",
|
|
"content": [
|
|
{"type": "text", "text": "Rule 1", "cache_control": {"type": "ephemeral", "ttl": "1h"}},
|
|
{"type": "text", "text": "\n\nAnswer with exactly one word."},
|
|
],
|
|
},
|
|
{"role": "user", "content": "What is the capital of France?"},
|
|
]
|
|
|
|
def test_non_consecutive_native_system_messages_stay_where_the_client_put_them(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": "System turn 1"},
|
|
{"role": "user", "content": "User turn 1"},
|
|
{"role": "system", "content": "System turn 2"},
|
|
{"role": "user", "content": "User turn 2"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == messages
|
|
|
|
def test_list_content_merge_preserves_block_level_cache_control(self) -> None:
|
|
messages: Final = [
|
|
{"role": "system", "content": [{"type": "text", "text": "Cached", "cache_control": {"type": "ephemeral"}}]},
|
|
{"role": "developer", "content": [{"type": "text", "text": "Extra"}]},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
assert list(hoist_developer_messages_into_leading_system_message(messages)) == [
|
|
{
|
|
"role": "system",
|
|
"content": [
|
|
{"type": "text", "text": "Cached", "cache_control": {"type": "ephemeral"}},
|
|
{"type": "text", "text": "\n\nExtra"},
|
|
],
|
|
},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|