mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-12 23:01:41 +00:00
fix(types): coerce dict server_tool_use to ServerToolUse in Usage init
Prerequisite for #31035 on this line, and a latent-bug fix in its own right. #31035's usage-only fallback builds server_tool_use as a dict and prices it via AnthropicConfig.calculate_usage, whose Usage(**model_dump()) round-trip drops it back to a plain dict; without this Usage.__init__ coercion the recovered-cost path does attribute access on a dict and raises, so #31035's web-search/server-tool cost recovery is dead on arrival here. The same round-trip already affected the pre-existing ChunkProcessor.calculate_usage path: every production consumer on this line (litellm/llms/anthropic/cost_calculation.py, litellm/litellm_core_utils/llm_cost_calc/tool_call_cost_tracking.py) reads usage.server_tool_use.web_search_requests by attribute, so a dict there is a latent AttributeError on streaming web-search cost. The coercion makes the value a ServerToolUse, which all consumers expect. Also updates the one test that pinned the old dict-subscript shape (test_stream_chunk_builder_anthropic_web_search) to assert the ServerToolUse type and attribute access, matching staging. Content-verified present on litellm_internal_staging via aggregatorf49707bc66(fix(otel) #30257), which carries both the coercion and the test assertion update; this restores only those, not the rest of that aggregator. The coercion also shipped to stable/1.89.x as24e30b551f. (cherry picked from commit24e30b551f)
This commit is contained in:
parent
c98c0dc1ba
commit
7ebb469551
2 changed files with 7 additions and 1 deletions
|
|
@ -1655,6 +1655,9 @@ class Usage(SafeAttributeModel, CompletionUsage):
|
|||
prompt_tokens_details=_prompt_tokens_details or None,
|
||||
)
|
||||
|
||||
if isinstance(server_tool_use, dict):
|
||||
server_tool_use = ServerToolUse(**server_tool_use)
|
||||
|
||||
if server_tool_use is not None:
|
||||
self.server_tool_use = server_tool_use
|
||||
else: # maintain openai compatibility in usage object if possible
|
||||
|
|
|
|||
|
|
@ -520,7 +520,10 @@ def test_stream_chunk_builder_anthropic_web_search():
|
|||
assert usage.prompt_tokens == 50
|
||||
assert usage.completion_tokens == 27
|
||||
assert usage.total_tokens == 77
|
||||
assert usage.server_tool_use["web_search_requests"] == 2
|
||||
# server_tool_use must be a ServerToolUse pydantic so downstream cost-calc
|
||||
# (which uses attribute access) works. See issue #26153.
|
||||
assert isinstance(usage.server_tool_use, ServerToolUse)
|
||||
assert usage.server_tool_use.web_search_requests == 2
|
||||
|
||||
|
||||
def test_sort_chunks_handles_dict_hidden_params_created_at():
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue