mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-09 03:18:44 +00:00
Merge 297a65d312 into e768ad55ce
This commit is contained in:
commit
cf27a629cc
12 changed files with 464 additions and 0 deletions
48
.github/copilot-instructions.md
vendored
Normal file
48
.github/copilot-instructions.md
vendored
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
# Copilot instructions for LiteLLM
|
||||
|
||||
This repo has detailed contributor docs already; read them rather than duplicating:
|
||||
|
||||
- `CLAUDE.md` / `AGENTS.md` (identical content) - coding conventions, lint/type-suppression rules, PR/commit conventions. **Read this in full before writing code.**
|
||||
- `ARCHITECTURE.md` - request flow diagrams for the SDK and the proxy (AI Gateway), the translation-layer pattern, and the data-access layer (`litellm/models/` + `litellm/repositories/`).
|
||||
- `CONTRIBUTING.md` - setup, testing, linting walkthrough.
|
||||
- `tests/test_litellm/readme.md` - unit test directory conventions.
|
||||
- `tests/e2e/CLAUDE.md` and `tests/e2e/CONTRIBUTING.md` - e2e harness rules (shared transport only, no raw `requests`, suite-per-folder layout).
|
||||
|
||||
## Big picture
|
||||
|
||||
LiteLLM is two things sharing one codebase:
|
||||
|
||||
1. **SDK** (`litellm/`): `main.py` (`completion()`/`acompletion()`) resolves provider via `utils.py:get_llm_provider()`, then `llms/custom_httpx/llm_http_handler.py` (`BaseLLMHTTPHandler`) calls a provider's `Config.transform_request()` / `transform_response()` (in `llms/{provider}/chat/transformation.py`, subclassing `BaseConfig` from `llms/base_llm/chat/transformation.py`). You almost never modify the handler itself, just the provider's `transformation.py`.
|
||||
2. **AI Gateway / Proxy** (`litellm/proxy/`): wraps the SDK with auth (`proxy/auth/user_api_key_auth.py`), rate limiting/budgets (`proxy/hooks/`), and routing (`router.py`), then calls into the SDK's `main.py` for the actual LLM call. Cost is calculated post-call in `litellm_logging.py` -> `cost_calculator.py` and queued to Postgres via `proxy/db/db_spend_update_writer.py`.
|
||||
|
||||
Persistence/caching: Redis (`caching/redis_cache.py`, `caching/dual_cache.py`) for rate limits/API-key cache/cooldowns; Postgres (`proxy/schema.prisma`) for keys/teams/users/spend logs, accessed through `litellm/repositories/` (`BaseRepository[T]` + entity repos) with entity Pydantic models in `litellm/models/` (re-exported from `proxy/_types.py` for backwards compat).
|
||||
|
||||
To add/modify a provider or feature, find the relevant `transformation.py` from the table in `ARCHITECTURE.md` section 3, and add unit tests in `tests/llm_translation/test_{provider}.py` that call `transform_request`/`transform_response` directly (no live API calls needed).
|
||||
|
||||
## Build / install
|
||||
|
||||
- Package manager is `uv`, not pip/poetry directly.
|
||||
- `make install-dev` - base dev deps. `make install-proxy-dev` - adds proxy extras. `make install-test-deps` - full env incl. Postgres/Prisma, needed for proxy or DB-backed tests.
|
||||
- `make bootstrap` - full fresh-clone/worktree setup (deps, Prisma client, UI npm install).
|
||||
|
||||
## Tests
|
||||
|
||||
- Unit tests: `tests/test_litellm/` mirrors `litellm/`'s structure 1:1 (e.g. `litellm/utils.py` -> `tests/test_litellm/test_utils.py`); **mocked only, no real API calls**.
|
||||
- Run one file: `uv run pytest tests/test_litellm/test_your_file.py -v`. Run one test: append `::test_name`.
|
||||
- Run everything: `make test-unit` (parallelized, `tests/test_litellm`). Other `make test-unit-*` targets split proxy/llms/integration tests into matrix groups mirroring CI (see `make help`).
|
||||
- Integration/live-API tests live under `tests/llm_translation/`, `tests/e2e/` (real proxy + real providers, see e2e CLAUDE.md/CONTRIBUTING.md), etc. — not mocked.
|
||||
- e2e tests must use the shared transport (`e2e_http.py`), never `requests.*` directly (enforced by CI check).
|
||||
|
||||
## Lint / type-check
|
||||
|
||||
- `make format` - ruff format (line length 120, not 88). `make lint` - full CI-parity lint (ruff, basedpyright budget gates, circular imports, import safety). `make lint-dev` - faster, changed-files-only version for local iteration.
|
||||
- basedpyright, ruff-strict, and type-discipline rules are gated by ratcheting budget files (`basedpyright-code-budget.json`, `ruff-strict-budget.json`, `type-discipline-budget.json`). If your change fixes violations, run `make lint-budget-update` and commit the lowered budgets.
|
||||
- `make pre-commit` runs the CI-equivalent checks on staged files and writes full output to a log file in `.git` (path printed first/last line) — read that log instead of re-running.
|
||||
- Every lint/type suppression must cite the exact rule and a reason, e.g. `# pyright: ignore[reportArgumentType] # ...`. `# type: ignore` is banned (disabled in `pyrightconfig.json`).
|
||||
|
||||
## Key conventions (see CLAUDE.md/AGENTS.md for full list)
|
||||
|
||||
- No mutation: don't reassign variables; annotate with `: Final`; build collections in one shot (comprehensions -> `tuple()`/`MappingProxyType()`/`frozenset()`) instead of seeding-and-mutating (`LIT001`/`LIT002`).
|
||||
- Fully typed; no bare `Any`/`dict`. Validate untyped inputs via Pydantic in the caller rather than loosening types.
|
||||
- Composition over inheritance; early returns over nesting; model failures as values rather than raising where a public contract exists.
|
||||
- Conventional Commits for commits/PR titles; branch off `litellm_internal_staging` (not `main`), branch names use `litellm_` prefix with no `/`.
|
||||
|
|
@ -1690,6 +1690,19 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
|
|||
"""
|
||||
return False
|
||||
|
||||
def should_strip_claude_code_identity(self) -> bool:
|
||||
"""
|
||||
Whether to drop Claude Code's self-identification from the system prompt.
|
||||
|
||||
Distinct from ``should_strip_billing_metadata``: that drops the billing header on
|
||||
every provider whose request shape rejects it, including Bedrock/Vertex/Azure, which
|
||||
still serve *Claude* models. This only trips on providers whose
|
||||
Anthropic-compatible endpoint serves a *different* model, where "You are Claude
|
||||
Code" is a false self-description. The first-party config and the Claude-serving
|
||||
providers keep it, so this stays False here.
|
||||
"""
|
||||
return False
|
||||
|
||||
def translate_system_message(self, messages: list[AllMessageValues]) -> list[AnthropicSystemMessageContent]:
|
||||
"""
|
||||
Translate system message to anthropic format.
|
||||
|
|
|
|||
|
|
@ -85,6 +85,9 @@ _CLAUDE_CODE_OBJECT_LIST_ADAPTER: Final = TypeAdapter(list[object])
|
|||
_CLAUDE_CODE_USER_AGENT_PREFIXES: Final = ("claude-cli/", "claude-code/")
|
||||
|
||||
|
||||
_CLAUDE_CODE_IDENTITY: Final = "You are Claude Code, Anthropic's official CLI for Claude."
|
||||
|
||||
|
||||
def requires_native_compaction_beta(
|
||||
custom_llm_provider: str,
|
||||
optional_params: Mapping[str, object],
|
||||
|
|
@ -127,6 +130,65 @@ def is_claude_code_user_agent(user_agent: str) -> bool:
|
|||
return user_agent.startswith(_CLAUDE_CODE_USER_AGENT_PREFIXES)
|
||||
|
||||
|
||||
def strip_claude_code_identity(text: str) -> str | None:
|
||||
"""Remove Claude Code's self-identification sentence from a single system text block.
|
||||
|
||||
Claude Code prefixes its system prompt with ``You are Claude Code, Anthropic's
|
||||
official CLI for Claude.``. When that prompt is forwarded to a non-Claude model
|
||||
(e.g. a DeepSeek-compatible or OpenAI-like Anthropic-compatible endpoint), the
|
||||
sentence is a false self-description and should be dropped before sending
|
||||
upstream. Returns ``None`` when the text was nothing but the identity sentence
|
||||
(so callers can drop the whole block), otherwise the text with the sentence and
|
||||
its framing whitespace removed.
|
||||
"""
|
||||
stripped: Final = text.strip()
|
||||
if stripped == _CLAUDE_CODE_IDENTITY:
|
||||
return None
|
||||
if stripped.startswith(_CLAUDE_CODE_IDENTITY):
|
||||
return text.replace(_CLAUDE_CODE_IDENTITY, "", 1).lstrip("\n")
|
||||
return text
|
||||
|
||||
|
||||
def strip_claude_code_identity_from_system(
|
||||
system_param: str | list[object] | None,
|
||||
) -> str | list[object] | None:
|
||||
"""Strip Claude Code's self-identification sentence from a full system parameter.
|
||||
|
||||
Unlike :func:`strip_claude_code_identity`, which operates on a single text
|
||||
block, this handles the whole ``system`` value of an Anthropic Messages
|
||||
request -- either a plain string or a list of system content blocks.
|
||||
Non-text blocks are kept as-is; text blocks have the identity sentence
|
||||
removed, and are dropped entirely when it was the only content.
|
||||
|
||||
Returns ``None`` when every block was dropped so callers can remove the whole
|
||||
``system`` parameter.
|
||||
"""
|
||||
if isinstance(system_param, str):
|
||||
return strip_claude_code_identity(system_param)
|
||||
if isinstance(system_param, list):
|
||||
filtered_list: Final = [] # mutable-ok: API message payload
|
||||
for content_block in system_param:
|
||||
if isinstance(content_block, dict):
|
||||
text = content_block.get("text", "")
|
||||
content_type = content_block.get("type", "")
|
||||
if content_type != "text":
|
||||
filtered_list.append(content_block)
|
||||
continue
|
||||
stripped_text = strip_claude_code_identity(text)
|
||||
if stripped_text is None:
|
||||
continue
|
||||
if stripped_text == text:
|
||||
filtered_list.append(content_block)
|
||||
else:
|
||||
rewritten = {**content_block, "text": stripped_text} # mutable-ok: API message payload
|
||||
filtered_list.append(rewritten)
|
||||
else:
|
||||
# Keep non-dict items as-is.
|
||||
filtered_list.append(content_block)
|
||||
return filtered_list if len(filtered_list) > 0 else None
|
||||
return system_param
|
||||
|
||||
|
||||
def _validated_claude_code_mapping(value: object) -> dict[object, object] | None:
|
||||
try:
|
||||
return _CLAUDE_CODE_OBJECT_MAPPING_ADAPTER.validate_python(value)
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ from typing import TYPE_CHECKING, Any, Final, Literal, TypeAlias, TypeVar, cast
|
|||
from pydantic import JsonValue, TypeAdapter
|
||||
|
||||
import litellm
|
||||
from litellm.llms.anthropic.common_utils import strip_claude_code_identity_from_system
|
||||
from litellm.llms.anthropic.pass_through.utils import (
|
||||
is_reasoning_auto_summary_enabled,
|
||||
prompt_cache_key_from_user_id,
|
||||
|
|
@ -1230,6 +1231,23 @@ class LiteLLMAnthropicMessagesAdapter:
|
|||
new_messages: list[AllMessageValues] = []
|
||||
tool_name_mapping: dict[str, str] = {}
|
||||
|
||||
# Strip Claude Code's self-identification from the system prompt before
|
||||
# translating to an OpenAI chat-completions request. This bridge only runs
|
||||
# for non-Anthropic models (first-party servers take the native
|
||||
# AnthropicMessagesConfig path), so "You are Claude Code" is always a
|
||||
# false self-description here and must not reach the target model.
|
||||
system_param: Final = anthropic_message_request.get("system")
|
||||
if system_param is not None:
|
||||
stripped_system = strip_claude_code_identity_from_system(system_param)
|
||||
if stripped_system is None or stripped_system != system_param:
|
||||
# This adapter is also invoked with a read-only wire-body mapping
|
||||
# (shadow evaluation); mutate a copy, never the caller's request.
|
||||
anthropic_message_request = cast(AnthropicMessagesRequest, dict(anthropic_message_request))
|
||||
if stripped_system is None:
|
||||
anthropic_message_request.pop("system", None)
|
||||
else:
|
||||
anthropic_message_request["system"] = stripped_system
|
||||
|
||||
## CONVERT ANTHROPIC MESSAGES TO OPENAI
|
||||
messages_list: Final[list[AllAnthropicPassThroughMessageValues]] = cast(
|
||||
list[AllAnthropicPassThroughMessageValues],
|
||||
|
|
|
|||
|
|
@ -28,6 +28,7 @@ from ...common_utils import (
|
|||
optionally_handle_anthropic_oauth,
|
||||
requires_native_compaction_beta,
|
||||
strip_advisor_blocks_from_messages,
|
||||
strip_claude_code_identity_from_system,
|
||||
strip_encrypted_reasoning_blocks_from_anthropic_messages,
|
||||
)
|
||||
from .mid_conversation_system import (
|
||||
|
|
@ -130,6 +131,32 @@ class AnthropicMessagesConfig(BaseAnthropicMessagesConfig):
|
|||
"""
|
||||
return False
|
||||
|
||||
def should_strip_claude_code_identity(self) -> bool:
|
||||
"""
|
||||
Whether to drop Claude Code's self-identification from the system prompt.
|
||||
|
||||
Distinct from ``should_strip_billing_metadata``: that drops the billing header on
|
||||
every provider whose request shape rejects it, including Bedrock/Vertex/Azure, which
|
||||
still serve *Claude* models. This only trips on providers whose
|
||||
Anthropic-compatible endpoint serves a *different* model (DeepSeek, MiniMax,
|
||||
Tencent, OpenAI-like passthrough), where "You are Claude Code" is a false
|
||||
self-description. The first-party config keeps it.
|
||||
"""
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
def _strip_claude_code_identity_from_system(system_param: str | list[object] | None) -> str | list[object] | None:
|
||||
"""
|
||||
Strip Claude Code's self-identification sentence from system parameter.
|
||||
|
||||
Args:
|
||||
system_param: Can be a string or a list of system message content blocks
|
||||
|
||||
Returns:
|
||||
System parameter with the identity sentence removed, or None if all content was removed
|
||||
"""
|
||||
return strip_claude_code_identity_from_system(system_param)
|
||||
|
||||
@staticmethod
|
||||
def _filter_billing_headers_from_system(system_param):
|
||||
"""
|
||||
|
|
@ -537,6 +564,14 @@ class AnthropicMessagesConfig(BaseAnthropicMessagesConfig):
|
|||
else:
|
||||
anthropic_messages_optional_request_params.pop("system", None)
|
||||
|
||||
stripped_identity_system: Final = anthropic_messages_optional_request_params.get("system")
|
||||
if self.should_strip_claude_code_identity() and stripped_identity_system is not None:
|
||||
identity_filtered: Final = self._strip_claude_code_identity_from_system(stripped_identity_system)
|
||||
if identity_filtered is not None and len(identity_filtered) > 0:
|
||||
anthropic_messages_optional_request_params["system"] = identity_filtered
|
||||
else:
|
||||
anthropic_messages_optional_request_params.pop("system", None)
|
||||
|
||||
# Transform context_management from OpenAI format to Anthropic format if needed
|
||||
context_management_param: Final = anthropic_messages_optional_request_params.get("context_management")
|
||||
if context_management_param is not None:
|
||||
|
|
|
|||
|
|
@ -29,6 +29,9 @@ class DeepSeekAnthropicMessagesConfig(AnthropicMessagesConfig):
|
|||
def should_strip_billing_metadata(self) -> bool:
|
||||
return True
|
||||
|
||||
def should_strip_claude_code_identity(self) -> bool:
|
||||
return True
|
||||
|
||||
@staticmethod
|
||||
def get_api_key(api_key: str | None = None) -> str | None:
|
||||
return api_key or get_secret_str("DEEPSEEK_API_KEY") or litellm.api_key
|
||||
|
|
|
|||
|
|
@ -31,6 +31,9 @@ class MinimaxMessagesConfig(AnthropicMessagesConfig):
|
|||
def should_strip_billing_metadata(self) -> bool:
|
||||
return True
|
||||
|
||||
def should_strip_claude_code_identity(self) -> bool:
|
||||
return True
|
||||
|
||||
@staticmethod
|
||||
def get_api_key(api_key: str | None = None) -> str | None:
|
||||
"""
|
||||
|
|
|
|||
|
|
@ -32,6 +32,9 @@ class OpenAILikeAnthropicMessagesConfig(AnthropicMessagesConfig):
|
|||
super().__init__()
|
||||
self._cache_control_ttl: Final = cache_control_ttl
|
||||
|
||||
def should_strip_claude_code_identity(self) -> bool:
|
||||
return True
|
||||
|
||||
def validate_anthropic_messages_environment(
|
||||
self,
|
||||
headers: dict[str, str],
|
||||
|
|
|
|||
|
|
@ -30,6 +30,9 @@ class TencentAnthropicMessagesConfig(AnthropicMessagesConfig):
|
|||
def should_strip_billing_metadata(self) -> bool:
|
||||
return True
|
||||
|
||||
def should_strip_claude_code_identity(self) -> bool:
|
||||
return True
|
||||
|
||||
@staticmethod
|
||||
def get_api_key(api_key: str | None = None) -> str | None:
|
||||
return api_key or get_secret_str("TENCENT_API_KEY") or litellm.api_key
|
||||
|
|
|
|||
|
|
@ -5832,6 +5832,118 @@ def test_should_strip_billing_metadata_by_provider(module_path, class_name, expe
|
|||
assert config_cls().should_strip_billing_metadata() is expected_strip
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"module_path, class_name, expected_strip",
|
||||
[
|
||||
# First-party and Claude-serving providers keep the identity sentence: the
|
||||
# model genuinely *is* Claude, even via Bedrock/Vertex/Azure.
|
||||
("litellm.llms.anthropic.chat.transformation", "AnthropicConfig", False),
|
||||
(
|
||||
"litellm.llms.anthropic.pass_through.messages.transformation",
|
||||
"AnthropicMessagesConfig",
|
||||
False,
|
||||
),
|
||||
(
|
||||
"litellm.llms.bedrock.claude_platform.transformation",
|
||||
"BedrockClaudePlatformConfig",
|
||||
False,
|
||||
),
|
||||
(
|
||||
"litellm.llms.vertex_ai.vertex_ai_partner_models.anthropic.transformation",
|
||||
"VertexAIAnthropicConfig",
|
||||
False,
|
||||
),
|
||||
("litellm.llms.azure_ai.anthropic.transformation", "AzureAnthropicConfig", False),
|
||||
# Non-Claude Anthropic-compatible endpoints strip the false self-description.
|
||||
("litellm.llms.minimax.messages.transformation", "MinimaxMessagesConfig", True),
|
||||
(
|
||||
"litellm.llms.deepseek.messages.transformation",
|
||||
"DeepSeekAnthropicMessagesConfig",
|
||||
True,
|
||||
),
|
||||
(
|
||||
"litellm.llms.tencent.messages.transformation",
|
||||
"TencentAnthropicMessagesConfig",
|
||||
True,
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_should_strip_claude_code_identity_by_provider(module_path, class_name, expected_strip):
|
||||
import importlib
|
||||
|
||||
config_cls = getattr(importlib.import_module(module_path), class_name)
|
||||
assert config_cls().should_strip_claude_code_identity() is expected_strip
|
||||
|
||||
|
||||
def _system_with_identity_block(identity_sentence: str) -> list:
|
||||
return [
|
||||
{
|
||||
"role": "system",
|
||||
"content": [
|
||||
# Claude Code sends the identity sentence as its own text block.
|
||||
{"type": "text", "text": identity_sentence},
|
||||
{"type": "text", "text": "real system prompt"},
|
||||
],
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_messages_request_strips_claude_code_identity_for_minimax():
|
||||
from litellm.llms.minimax.messages.transformation import MinimaxMessagesConfig
|
||||
from litellm.types.router import GenericLiteLLMParams
|
||||
|
||||
config = MinimaxMessagesConfig()
|
||||
assert config.should_strip_claude_code_identity() is True
|
||||
|
||||
optional_params = {
|
||||
"max_tokens": 16,
|
||||
"system": [
|
||||
{"type": "text", "text": "You are Claude Code, Anthropic's official CLI for Claude."},
|
||||
{"type": "text", "text": "real system prompt"},
|
||||
],
|
||||
}
|
||||
result = config.transform_anthropic_messages_request(
|
||||
model="MiniMax-M2",
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
anthropic_messages_optional_request_params=optional_params,
|
||||
litellm_params=GenericLiteLLMParams(),
|
||||
headers={},
|
||||
)
|
||||
|
||||
texts = [block["text"] for block in result.get("system", [])]
|
||||
assert "You are Claude Code, Anthropic's official CLI for Claude." not in texts
|
||||
assert "real system prompt" in texts
|
||||
|
||||
|
||||
def test_messages_request_keeps_claude_code_identity_for_first_party():
|
||||
from litellm.llms.anthropic.pass_through.messages.transformation import (
|
||||
AnthropicMessagesConfig,
|
||||
)
|
||||
from litellm.types.router import GenericLiteLLMParams
|
||||
|
||||
config = AnthropicMessagesConfig()
|
||||
assert config.should_strip_claude_code_identity() is False
|
||||
|
||||
optional_params = {
|
||||
"max_tokens": 16,
|
||||
"system": [
|
||||
{"type": "text", "text": "You are Claude Code, Anthropic's official CLI for Claude."},
|
||||
{"type": "text", "text": "real system prompt"},
|
||||
],
|
||||
}
|
||||
result = config.transform_anthropic_messages_request(
|
||||
model="claude-3-5-sonnet-latest",
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
anthropic_messages_optional_request_params=optional_params,
|
||||
litellm_params=GenericLiteLLMParams(),
|
||||
headers={},
|
||||
)
|
||||
|
||||
texts = [block["text"] for block in result.get("system", [])]
|
||||
assert "You are Claude Code, Anthropic's official CLI for Claude." in texts
|
||||
assert "real system prompt" in texts
|
||||
|
||||
|
||||
def test_namespace_tool_flat_nested_tools_are_extracted():
|
||||
"""Codex sends nested tools in flat format {type, name, description, parameters} with no 'function' wrapper.
|
||||
These must be normalized and mapped without raising KeyError: 'function'."""
|
||||
|
|
@ -6379,6 +6491,29 @@ def test_eager_input_streaming_reaches_anthropic_request_tools():
|
|||
assert result["tools"][0]["name"] == "write_file"
|
||||
|
||||
|
||||
def test_messages_request_strips_identity_only_system_for_minimax():
|
||||
"""Identity-only system drops the whole system param (pop path)."""
|
||||
from litellm.llms.minimax.messages.transformation import MinimaxMessagesConfig
|
||||
from litellm.types.router import GenericLiteLLMParams
|
||||
|
||||
config = MinimaxMessagesConfig()
|
||||
optional_params = {
|
||||
"max_tokens": 16,
|
||||
"system": [
|
||||
{"type": "text", "text": "You are Claude Code, Anthropic's official CLI for Claude."},
|
||||
],
|
||||
}
|
||||
result = config.transform_anthropic_messages_request(
|
||||
model="MiniMax-M2",
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
anthropic_messages_optional_request_params=optional_params,
|
||||
litellm_params=GenericLiteLLMParams(),
|
||||
headers={},
|
||||
)
|
||||
|
||||
assert "system" not in result
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Mid-conversation ``role: "system"`` on the chat completions path.
|
||||
#
|
||||
|
|
|
|||
|
|
@ -2862,6 +2862,68 @@ def test_translate_completion_input_params_keeps_provider_native_tools():
|
|||
assert translated["tools"] == [{"googleMaps": {}}]
|
||||
|
||||
|
||||
CLAUDE_CODE_IDENTITY = "You are Claude Code, Anthropic's official CLI for Claude."
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"system,expected_system_content",
|
||||
[
|
||||
# Identity sentence alone -> the whole system message is dropped. The bridge
|
||||
# only serves non-Anthropic models, so the first-party case is never here.
|
||||
(CLAUDE_CODE_IDENTITY, None),
|
||||
# Identity followed by the real prompt -> the real prompt survives.
|
||||
(f"{CLAUDE_CODE_IDENTITY}\nYou are an interactive agent.", "You are an interactive agent."),
|
||||
# Non-identity system prompt is passed through untouched.
|
||||
("You are a helpful assistant.", "You are a helpful assistant."),
|
||||
],
|
||||
)
|
||||
def test_translate_anthropic_to_openai_strips_claude_code_identity(system, expected_system_content):
|
||||
"""The chat-completions bridge must not forward Claude Code's self-identification upstream."""
|
||||
from litellm.types.llms.anthropic import AnthropicMessagesRequest
|
||||
|
||||
adapter = LiteLLMAnthropicMessagesAdapter()
|
||||
openai_request, _ = adapter.translate_anthropic_to_openai(
|
||||
anthropic_message_request=AnthropicMessagesRequest(
|
||||
model="hosted_vllm/kimi-k3",
|
||||
max_tokens=1024,
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
system=system,
|
||||
),
|
||||
custom_llm_provider="hosted_vllm",
|
||||
)
|
||||
|
||||
system_messages = [m for m in openai_request["messages"] if m["role"] == "system"]
|
||||
if expected_system_content is None:
|
||||
assert system_messages == []
|
||||
else:
|
||||
assert len(system_messages) == 1
|
||||
assert system_messages[0]["content"] == expected_system_content
|
||||
|
||||
|
||||
def test_translate_anthropic_to_openai_strips_claude_code_identity_read_only_request():
|
||||
"""Identity stripping must not mutate a read-only request (shadow evaluation passes a
|
||||
MappingProxyType, so the adapter must copy before rewriting ``system``)."""
|
||||
from types import MappingProxyType
|
||||
|
||||
adapter = LiteLLMAnthropicMessagesAdapter()
|
||||
request = MappingProxyType(
|
||||
{
|
||||
"model": "hosted_vllm/kimi-k3",
|
||||
"max_tokens": 1024,
|
||||
"messages": [{"role": "user", "content": "hi"}],
|
||||
"system": f"{CLAUDE_CODE_IDENTITY}\nYou are an interactive agent.",
|
||||
}
|
||||
)
|
||||
|
||||
openai_request, _ = adapter.translate_anthropic_to_openai(
|
||||
anthropic_message_request=request,
|
||||
custom_llm_provider="hosted_vllm",
|
||||
)
|
||||
|
||||
system_messages = [m for m in openai_request["messages"] if m["role"] == "system"]
|
||||
assert [m["content"] for m in system_messages] == ["You are an interactive agent."]
|
||||
|
||||
|
||||
def test_translate_openai_content_to_anthropic_reasoning_content_without_thinking_blocks():
|
||||
"""
|
||||
Test that reasoning_content is converted to thinking block when thinking_blocks is not present.
|
||||
|
|
|
|||
|
|
@ -54,6 +54,85 @@ def test_is_claude_code_one_shot_subagent_request(messages, system, expected):
|
|||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text,expected",
|
||||
[
|
||||
# Identity sentence alone -> drop the whole block.
|
||||
("You are Claude Code, Anthropic's official CLI for Claude.", None),
|
||||
# Identity sentence followed by the rest of the system prompt -> keep the rest.
|
||||
(
|
||||
"You are Claude Code, Anthropic's official CLI for Claude.\nYou are an interactive agent.",
|
||||
"You are an interactive agent.",
|
||||
),
|
||||
# Leading/trailing whitespace around the bare sentence is treated as drop-only.
|
||||
(" You are Claude Code, Anthropic's official CLI for Claude. ", None),
|
||||
# Non-Claude-Code prompts are untouched.
|
||||
("You are a helpful assistant.", "You are a helpful assistant."),
|
||||
# A sentence that merely mentions Claude Code is untouched.
|
||||
("You are Claude Code's helper.", "You are Claude Code's helper."),
|
||||
],
|
||||
)
|
||||
def test_strip_claude_code_identity(text, expected):
|
||||
from litellm.llms.anthropic.common_utils import strip_claude_code_identity
|
||||
|
||||
assert strip_claude_code_identity(text) == expected
|
||||
|
||||
|
||||
CLAUDE_CODE_IDENTITY = "You are Claude Code, Anthropic's official CLI for Claude."
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"system_param,expected",
|
||||
[
|
||||
# String form: identity sentence alone -> drop the whole system param.
|
||||
(CLAUDE_CODE_IDENTITY, None),
|
||||
# String form: identity followed by the real prompt -> keep the real prompt.
|
||||
(f"{CLAUDE_CODE_IDENTITY}\nYou are an interactive agent.", "You are an interactive agent."),
|
||||
# String form: non-identity text is untouched.
|
||||
("You are a helpful assistant.", "You are a helpful assistant."),
|
||||
# List form: identity text block is dropped, non-text blocks and the
|
||||
# surviving text block are preserved.
|
||||
(
|
||||
[
|
||||
{"type": "text", "text": CLAUDE_CODE_IDENTITY},
|
||||
{"type": "text", "text": "real system prompt"},
|
||||
],
|
||||
[{"type": "text", "text": "real system prompt"}],
|
||||
),
|
||||
# List form: identity-only text means every block is dropped.
|
||||
([{"type": "text", "text": CLAUDE_CODE_IDENTITY}], None),
|
||||
# List form: non-text blocks survive identity stripping untouched.
|
||||
(
|
||||
[
|
||||
{"type": "text", "text": CLAUDE_CODE_IDENTITY},
|
||||
{"type": "text", "text": "real system prompt", "cache_control": {"type": "ephemeral"}},
|
||||
],
|
||||
[{"type": "text", "text": "real system prompt", "cache_control": {"type": "ephemeral"}}],
|
||||
),
|
||||
# List form: identity + real content in the same block -> rewritten block.
|
||||
(
|
||||
[{"type": "text", "text": f"{CLAUDE_CODE_IDENTITY}\nreal system prompt"}],
|
||||
[{"type": "text", "text": "real system prompt"}],
|
||||
),
|
||||
# List form: a non-text block (type "other") is kept as-is.
|
||||
(
|
||||
[{"type": "text", "text": CLAUDE_CODE_IDENTITY}, {"type": "other", "foo": "bar"}],
|
||||
[{"type": "other", "foo": "bar"}],
|
||||
),
|
||||
# List form: non-dict items are kept as-is.
|
||||
([{"type": "text", "text": CLAUDE_CODE_IDENTITY}, "loose item"], ["loose item"]),
|
||||
# Non-string, non-list system payload is passed through untouched.
|
||||
(None, None),
|
||||
],
|
||||
)
|
||||
def test_strip_claude_code_identity_from_system(system_param, expected):
|
||||
from litellm.llms.anthropic.common_utils import (
|
||||
strip_claude_code_identity_from_system,
|
||||
)
|
||||
|
||||
assert strip_claude_code_identity_from_system(system_param) == expected
|
||||
|
||||
|
||||
class TestOptionallyHandleAnthropicOAuth:
|
||||
"""Tests for optionally_handle_anthropic_oauth function."""
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue