* feat(proxy): log one line per token count call
Each count call on /v1/messages/count_tokens, /utils/token_counter, /v1/responses/input_tokens and the Gemini countTokens routes now logs one INFO line with the litellm call id, the requested model, the model the count ran against, the count, the count source and the status code, so it joins the inference call that sends the same x-litellm-call-id
* test(google_endpoints): give the count test's fake request a headers list
* fix(proxy): keep token count log lines one per call and trusted in JSON logs, and log Bedrock native counts
Quote log values that carry whitespace, quotes, equals, or control characters so a model name can't split or forge a line. Let JsonFormatter record extras win over a dict parsed from the message. Log the Bedrock native count-tokens route through the same line.
* fix(proxy): escape braces in token count log values, log the Bedrock model AWS counts, and skip the line when there is nothing to count
* fix(proxy): log the deployment id the count routed to on every token count line
* fix(proxy): log count requests rejected before counting with the status they answer
A count route that rejects a request before the shared count function runs now logs the same token count line with its 400. An unsupported content block on the Messages count route becomes its 400 inside the logged wrapper, so its line says 400 instead of 500. A count that ends in a BaseException other than cancellation logs a 500.
* fix(proxy): compute the Bedrock counted model id only after a successful count
* revert(logging): keep the JSON log formatter's field order as on main
The count line already escapes braces in its values, so the formatter never parses a dict from it. Changing the order for every logger is a separate decision.
* fix: align token counting with inference context
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: preserve Vertex Gemini count message shapes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: forward Google countTokens tools in shapes every provider counter accepts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: add package marker for Vertex count-token unit tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): ignore deployment system_instruction when counting tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(count_tokens): forward system and tools on Google AI Studio, keep strict on Anthropic tools
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(count_tokens): ruff format gemini count handler
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(count_tokens): return Mapping from count request body builders
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(count_tokens): truncate long Anthropic tool names on Vertex count, update stale strict test expectations
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(count_tokens): simplify Gemini count context and tests
Put the Vertex Gemini count branch back in the same shape as main and the partner model branch, keeping only the system and tools context, the provider error mapping and the gemini tokenizer default
Share one error response helper between the Vertex and AI Studio callers, drop the unread kind fields and the extra validation of map_function output, and say Gemini instead of Vertex AI in the error that AI Studio users also see
Tighten the tests by removing a stray parametrize on a helper, sharing the SDK count args, asserting the exact forwarded system and tools, and folding the Anthropic tool shape tests into one table
* fix(count_tokens): reuse inference tool translation for Vertex Gemini count
Vertex and AI Studio count now convert tools with the same functions inference uses, translate_anthropic_tools_to_openai and the Responses tool converter, before VertexGeminiConfig.map_function. Unnamed and Anthropic hosted tools now count the same declarations inference sends instead of being dropped
Tools that cannot be converted, such as a non list tools value, a non object tool or a schema that is not a JSON object, now return a 400 before any request instead of being counted without them
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): move utils, agent_endpoints and endpoint tests into tests/unit/proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): package moved unit test directories
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): exclude proxy-db-owned files from the misc target
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): drop the redundant fixture docstrings in the proxy conftest
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-01 11:06:42 -07:00
Renamed from tests/test_litellm/proxy/google_endpoints/test_endpoints.py (Browse further)
