mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-06 08:16:43 +00:00
81277252e1
The LLM Obs callback copied litellm's OpenAI-shaped objects into the span verbatim, so every field Datadog names differently landed somewhere it does not read: tool calls kept their nested `function` wrapper instead of DD's name/arguments/tool_id, tool messages carried no result linking them to their call, the request's tools were never sent, and prompt-cache counts sat inside meta.metadata rather than the span metrics its cache dashboards chart. One rule governs the message mapper: add the fields Datadog declares, and never destroy content it did not understand. Content collapses to its text only when it has text, so a content list carrying tool or image blocks rides along unchanged, and absent messages map to an empty input rather than a fabricated turn. Tool calls and results are read from both dialects, the OpenAI `tool_calls` / `role: tool` shape and the Anthropic `tool_use` / `tool_result` content blocks, so /v1/messages sessions gain tool linking they never had. Cache counts come from the same owners the savings dashboard uses, so every provider spelling resolves through one place rather than a second local guess. The three cache metrics partition the input count: litellm's normalized prompt total includes both cache categories, as the cost calculator's pricing helper documents, so the non-cached residual subtracts reads AND writes. Counting a primed prefix as ordinary input had inflated non-cached usage by exactly the cache-write count on every priming request. Correlating a result to its call reads ids and names structurally and parses no arguments, so a tool call's arguments are decoded once per span rather than once per pass, and arguments past a size bound ship as the raw string instead of paying a decode that multiplies memory on hostile compact JSON. The flat `output_tool_calls.*` metadata copies go away with this: they were a second representation of a fact that now has its own field on the same span. |
||
|---|---|---|
| .. | ||
| test_datadog_cost_management.py | ||
| test_datadog_llm_obs.py | ||
| test_datadog_llm_obs_agent.py | ||
| test_datadog_logger_batching.py | ||
| test_datadog_metrics.py | ||
| test_datadog_tags_regression.py | ||
| test_datadog_team_handler.py | ||