mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-12 03:48:56 +00:00
50ab9a3211
* fix(token_counter): price a base64 PDF document per page instead of as one image A `document` or `file` block carrying inline PDF bytes was priced like a single image (85 tokens), so when a provider's count-tokens endpoint rejected the model (Bedrock Opus) the local fallback answered 116 for a 12-page PDF the provider then billed at 35941 input tokens. The counter now reads the PDF with pypdf and prices each page as its extracted text plus the image Anthropic renders it to (1568 px long edge, 1.15 MP, 750 pixels per token), falling back to the old image pricing when pypdf is missing or the bytes are not a readable PDF. * refactor(token_counter): count PDF pages as they are read Sum each page's text and rendered-image tokens straight from the pypdf reader instead of materializing a page list first, keep the fallback to image pricing atomic when a page cannot be read, and annotate the new tests' locals as Final * test(integration): audit cells for page-priced PDF documents in count_tokens, pre-call checks and spend * test(integration): release the held peer when the concurrent budget wait times out --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| either_role.json | ||
| test_advisor_failure_cooldown.py | ||
| test_audit_any_sweep_chaos.py | ||
| test_complexity_router_llm_classifier.py | ||
| test_deployment_num_retries_generic_routes_wire.py | ||
| test_encrypted_content_affinity_no_peer.py | ||
| test_end_user_region_routing.py | ||
| test_fallback_hop_foreign_encrypted_reasoning.py | ||
| test_fallback_retry_header_contracts.py | ||
| test_include_fallback_errors_wire.py | ||
| test_key_max_parallel_requests.py | ||
| test_key_tpm_reservation.py | ||
| test_messages_stream_retry_budget_sources_owned_proxy.py | ||
| test_observed_routing.py | ||
| test_payment_required_cooldown.py | ||
| test_payment_required_mapping.py | ||
| test_pdf_document_pre_call_checks_owned_proxy.py | ||
| test_priority_model_tpm_enforcement.py | ||
| test_priority_rate_limit_headers.py | ||
| test_priority_scheduler_queue_cleanup.py | ||
| test_prompt_caching_affinity_tool_call_marks.py | ||
| test_redis_increment_with_floor.py | ||
| test_redis_recovery.py | ||
| test_stale_cost_map_boot.py | ||
| test_team_model_tpm_limit.py | ||
| test_team_tag_routing.py | ||
| test_usage_based_routing_redis_reads.py | ||
| test_user_config_routing.py | ||