litellm/tests/integration/_support
devin-ai-integration[bot] 50ab9a3211
fix(token_counter): price a base64 PDF document per page instead of as one image (#45301)
* fix(token_counter): price a base64 PDF document per page instead of as one image

A `document` or `file` block carrying inline PDF bytes was priced like a single
image (85 tokens), so when a provider's count-tokens endpoint rejected the
model (Bedrock Opus) the local fallback answered 116 for a 12-page PDF the
provider then billed at 35941 input tokens. The counter now reads the PDF with
pypdf and prices each page as its extracted text plus the image Anthropic
renders it to (1568 px long edge, 1.15 MP, 750 pixels per token), falling back
to the old image pricing when pypdf is missing or the bytes are not a readable PDF.

* refactor(token_counter): count PDF pages as they are read

Sum each page's text and rendered-image tokens straight from the pypdf
reader instead of materializing a page list first, keep the fallback
to image pricing atomic when a page cannot be read, and annotate the
new tests' locals as Final

* test(integration): audit cells for page-priced PDF documents in count_tokens, pre-call checks and spend

* test(integration): release the held peer when the concurrent budget wait times out

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-08 21:20:46 -07:00
..
__init__.py test: add CircleCI integration contract foundation 2026-09-14 03:30:52 -07:00
agentic_probe.py fix(responses): honor caller stream flag when provider forces SSE (internal copy of #34095) (#41235) 2026-10-06 03:30:47 +00:00
anthropic_sse.py fix(router): retry a /v1/messages stream the provider drops before the first content chunk (#44276) 2026-10-05 22:53:39 +00:00
anthropic_thinking.py fix(anthropic): stop repeating streamed thinking text in the signature chunk (#44127) 2026-10-02 19:38:41 -07:00
asgi.py test(integration): add MCP gateway coverage wave 1 with a dedicated mcp shard and proxy coverage artifact (#42711) 2026-09-23 07:48:46 -07:00
bedrock_runtime_peer.py test(ci): fix six CircleCI test regressions on main (#44429) 2026-10-03 20:55:01 -07:00
browser_state.py test: add extension and browser integration contracts 2026-09-16 13:15:53 -07:00
claude_code.py test(anthropic): native /v1/messages reasoning integration tests built on a captured Claude Code request (#43361) 2026-10-01 09:12:53 -07:00
client.py test(ci): repair stale tests and flaky CI infrastructure (#43983) 2026-10-01 17:46:43 +00:00
codex_vendor.py fix(responses): honor caller stream flag when provider forces SSE (internal copy of #34095) (#41235) 2026-10-06 03:30:47 +00:00
daily_activity.py fix(proxy): look up hashed key names with two spend log rows per key (#43656) 2026-09-29 20:21:55 -07:00
daily_spend_rows.py fix(ui): group cost optimization cache leakage by model group (#43008) 2026-09-24 23:51:51 -07:00
database.py test(integration): run the Langfuse DB-callback test on its own scratch database (#43288) 2026-09-26 00:15:56 -07:00
database_relay.py fix(spend-tracking): stop caching failed spend-log metadata lookups as confirmed misses (#43560) 2026-10-05 22:36:25 +00:00
device_login.py fix(chatgpt,github_copilot): refuse device-code login inside an event loop or worker thread (#39585) 2026-10-06 03:20:07 +00:00
forward_proxy.py feat(voyage): rebrand to VoyageAI by MongoDB and route MongoDB keys to ai.mongodb.com (#41812) 2026-10-08 13:12:47 -07:00
generation.py fix(tests): resolve the integration support package without run.py's PYTHONPATH 2026-09-15 21:10:08 -07:00
mail.py feat(proxy): email alerts at configured percentages of a team member budget (#42665) 2026-09-26 02:42:45 +00:00
manifest.py fix(caching): never store or serve a chat completion with no choices (#44709) 2026-10-06 20:07:30 +00:00
mcp.py fix(mcp): honor scoped cache freshness (#45165) 2026-10-07 17:19:24 -07:00
mcp_grants.py fix(mcp): resolve team-granted toolsets for non-admin keys and dashboard sessions (#43908) 2026-10-01 14:15:06 -07:00
mcp_stdio_peer.py test(integration): add MCP gateway coverage wave 1 with a dedicated mcp shard and proxy coverage artifact (#42711) 2026-09-23 07:48:46 -07:00
oauth_server.py test(integration): credential canary slots for MCP and pass-through credentials (#43308) 2026-09-28 17:10:51 -07:00
openai_wire.py test: fix shared-provider discovery, Codex catalog size and generated master key mismatches in CircleCI suites (#44905) 2026-10-07 00:11:25 +00:00
otlp_sink.py feat(otel v2): excluded_services opt-out for datastore spans on tenant destinations (#43278) 2026-09-30 12:20:27 -07:00
pdf_document.py fix(token_counter): price a base64 PDF document per page instead of as one image (#45301) 2026-10-08 21:20:46 -07:00
process.py test(integration): boot owned proxies with a readiness-sized worker healthcheck budget (#45292) 2026-10-07 23:40:26 -07:00
prometheus_series.py feat(prometheus): cap series per metric for every labeled metric (#44420) 2026-10-06 01:28:44 +00:00
prompt_cache_breakpoint.py fix(responses): map input_audio blocks in the chat-to-Responses bridge (#45224) 2026-10-07 22:25:35 -07:00
provider.py test: fix shared-provider discovery, Codex catalog size and generated master key mismatches in CircleCI suites (#44905) 2026-10-07 00:11:25 +00:00
proxy.py test(ci): repair stale tests and flaky CI infrastructure (#43983) 2026-10-01 17:46:43 +00:00
redis_process.py test: repair stale and polluting tests red on scheduled main CI (#44229) 2026-10-02 21:32:17 +00:00
responses_stream.py fix(streaming): stop re-wrapping a bridged stream's MidStreamFallbackError (#44989) 2026-10-07 16:01:03 -07:00
responses_vendor.py fix(responses): drop bridge-minted reasoning items from OpenAI replays (#44132) 2026-10-02 19:39:48 -07:00
routing.py test(integration): add read-replica routing harness to the CircleCI integration suite (#42692) 2026-09-24 00:25:03 -07:00
sigv4.py test: cover provider wire contracts, streaming and recovery 2026-09-16 12:48:30 -07:00
tls.py fix(azure_storage): name Data Lake objects without base64 padding or slashes (#43914) 2026-09-30 18:17:13 -07:00
tool_rows.py feat(tool-policies): show the user who owns the key that discovered a tool (#43892) 2026-10-01 20:34:57 +00:00
upstream.py fix(vertex_ai): dial the multi-region Live API host for realtime sessions (#45166) 2026-10-07 17:39:38 -07:00
vertex.py fix(vertex_ai): forward system and tools to partner model count_tokens (#43900) 2026-10-03 16:36:14 -07:00
wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00