litellm/tests/claude_code/sample_compatibility-matrix.json
mateo-berri c1955a15ca RALPH: fix compat matrix - Azure now hosts Claude via Microsoft Foundry
Anthropic and Microsoft announced Claude Haiku 4.5, Sonnet 4.5/4.6, and
Opus 4.1/4.6/4.7 in Microsoft Foundry on 2025-11-18, so the matrix's
Azure column should exercise a real route through the LiteLLM proxy
rather than report not_applicable.

Foundry serves Claude on an Anthropic-shape /anthropic/v1/messages
endpoint (not the Azure OpenAI chat-completions route), and LiteLLM
already supports it via the azure_ai/claude-* provider prefix
(litellm/llms/azure_ai/anthropic/{handler,transformation,messages_transformation}.py).

- test_config.yaml: add 3 azure aliases pointing at azure_ai/claude-*
  with AZURE_FOUNDRY_API_BASE / AZURE_FOUNDRY_API_KEY env
- 6x test_azure.py: replace not_applicable stubs with real run_claude
  drivers, mirroring the existing test_vertex_ai.py shape exactly
- sample_compatibility-matrix.json: Azure cells flip to pass
- _builder_unit_tests: pin the new invariant (run_claude is used,
  not_applicable is gone) and feed pass results across all 5 providers
  in the 6x5 golden test
2026-04-25 17:58:32 -07:00

141 lines
2.8 KiB
JSON

{
"schema_version": "1",
"generated_at": "2026-04-25T00:00:00Z",
"litellm_version": "v1.83.0-stable",
"claude_code_version": "2.1.120",
"providers": [
"anthropic",
"bedrock_invoke",
"bedrock_converse",
"vertex_ai",
"azure"
],
"features": [
{
"id": "basic_messaging_non_streaming",
"name": "Basic messaging (non-streaming)",
"providers": {
"anthropic": {
"status": "pass"
},
"bedrock_invoke": {
"status": "pass"
},
"bedrock_converse": {
"status": "pass"
},
"vertex_ai": {
"status": "pass"
},
"azure": {
"status": "pass"
}
}
},
{
"id": "basic_messaging_streaming",
"name": "Basic messaging (streaming)",
"providers": {
"anthropic": {
"status": "pass"
},
"bedrock_invoke": {
"status": "pass"
},
"bedrock_converse": {
"status": "pass"
},
"vertex_ai": {
"status": "pass"
},
"azure": {
"status": "pass"
}
}
},
{
"id": "tool_use",
"name": "Tool use",
"providers": {
"anthropic": {
"status": "pass"
},
"bedrock_invoke": {
"status": "pass"
},
"bedrock_converse": {
"status": "pass"
},
"vertex_ai": {
"status": "pass"
},
"azure": {
"status": "pass"
}
}
},
{
"id": "prompt_caching_5m",
"name": "Prompt caching (5m TTL)",
"providers": {
"anthropic": {
"status": "pass"
},
"bedrock_invoke": {
"status": "pass"
},
"bedrock_converse": {
"status": "pass"
},
"vertex_ai": {
"status": "pass"
},
"azure": {
"status": "pass"
}
}
},
{
"id": "vision",
"name": "Vision",
"providers": {
"anthropic": {
"status": "pass"
},
"bedrock_invoke": {
"status": "pass"
},
"bedrock_converse": {
"status": "pass"
},
"vertex_ai": {
"status": "pass"
},
"azure": {
"status": "pass"
}
}
},
{
"id": "extended_thinking",
"name": "Extended thinking",
"providers": {
"anthropic": {
"status": "pass"
},
"bedrock_invoke": {
"status": "pass"
},
"bedrock_converse": {
"status": "pass"
},
"vertex_ai": {
"status": "pass"
},
"azure": {
"status": "pass"
}
}
}
]
}