Commit graph

53252 commits

Author SHA1 Message Date
Kerry Lu
5684267b12 fix(logging): report the cost-map entry billing used in model_map_information (#47182)
Backport of #47182 to stable/1.104.x.
Cherry-picked from 1212260ab1 (main).
Adapted to the line's private helper names (_select_model_name_for_cost_calc, _response_cost_calculator); the new test is appended to the line's test file.
2026-10-20 16:12:01 -07:00
Paul El Khoury
54f9259b40 fix(openai): drop unsupported reasoning effort for gpt-6.1-sol (#43935)
Backport of #43935 to stable/1.104.x.
Cherry-picked from d7620c77a1 (main).
Adapted to the line: the bridge check lives in litellm/main.py rather than litellm/responses/bridge_check.py, the effort helpers keep the line's private names, and the azure gpt-6.1-sol rows get supports_none_reasoning_effort=false (from #46377 on main) since this fix and its tests key on it. Test files keep the line's content plus this PR's own hunks.
2026-10-20 16:12:01 -07:00
devin-ai-integration[bot]
93faf0a1d5 chore(cost-map): add azure and openrouter gpt-6.1-sol rows (#43744)
Backport of #43744 to stable/1.104.x.
Cherry-picked from bda2763f2c (main).
Prerequisite for #43935: adds the azure and openrouter gpt-6.1-sol cost-map rows that fix gates on. Appended to the line's cost map.
2026-10-20 16:12:01 -07:00
berriai-litellm-provider-info-sync[bot]
3786195eb4 chore(cost-map): add openai gpt-6.1-sol from the pricing page (#43738)
Backport of #43738 to stable/1.104.x.
Cherry-picked from a3552c451b (main).
Prerequisite for #43935: adds the gpt-6.1-sol cost-map row that fix gates on. Appended to the line's cost map.
2026-10-20 16:12:01 -07:00
joshua-berri
f7b0b09e75 fix(batches): preserve spend across malformed output and metadata failures (#47184)
Backport of #47184 to stable/1.104.x.
Cherry-picked from 3fb95c8fe3 (main).
Ported by hand onto the line's poller, whose surrounding code predates several main-only batch features: the claim now writes the terminal status and NUL-stripped file_object, a failed spend-log write restores both, and the managed-files postgres_storable_json sanitizer (from #45703 on main) is added to the line since this fix needs it.
2026-10-20 16:12:01 -07:00
joshua-berri
bb1e40c3e5 fix(batches): handle empty models when logging batch cost (#47155)
Backport of #47155 to stable/1.104.x.
Cherry-picked from 4143e0869e (main).
2026-10-20 16:12:01 -07:00
devin-ai-integration[bot]
8dc91d9d0b fix(rust): add the dangerous-tool-use beta when an anthropic /v1/messages request carries safeguards (#46102)
Backport of #46102 to stable/1.104.x.
Cherry-picked from 5b042c0db6 (main).
Ported to the line's Rust messages layer for the anthropic provider only: a non-null safeguards field adds dangerous-tool-use-2026-09-03. Azure AI keeps its current headers since its beta-map cell is null on this line, and the other Rust providers and integration tests in the source are not on this line.
2026-10-20 16:12:01 -07:00
devin-ai-integration[bot]
2ac8b35a05 fix(model_prices): consolidate claude-haiku-5-5 over-100k pricing and capability flags (#45151)
Backport of #45151 to stable/1.104.x.
Cherry-picked from 46d440ae96 (main).
Conflicts resolved by keeping the line's fields and adding only the over-100k fields this PR introduces; cost-map entries replaced per model.
2026-10-20 16:12:00 -07:00
Krrish Dholakia
d3366b93b4 feat(model_prices): add claude-haiku-5-5 model pricing (#45108)
Backport of #45108 to stable/1.104.x.
Cherry-picked from 5cd1126870 (main).
Prerequisite for #45151: adds the claude-haiku-5-5 entries that fix corrects. Cost-map conflicts resolved by appending the new entries.
2026-10-20 16:12:00 -07:00
devin-ai-integration[bot]
61c5d953ee fix(anthropic): add the dangerous-tool-use beta when a /v1/messages request carries safeguards (#43955)
Backport of #43955 to stable/1.104.x.
Cherry-picked from eb5295b20c (main).
Adapted to the line: the transformation lives under anthropic/experimental_pass_through, the integration tests live under tests/integration/providers, and the CLI-beta wire assertion change is dropped since that test is not on the line.
2026-10-20 16:12:00 -07:00
mrinal-berri
2f19d8ed87 fix(databricks): send a single plain text block as a string for non-Claude models (#45686)
Backport of #45686 to stable/1.104.x.
Cherry-picked from 1aee1a21ed (main).
Test-file import conflict resolved by keeping the line's imports plus this PR's own.
2026-10-20 15:32:54 -07:00
mrinal-berri
6ba8151a2d fix(databricks): keep #/$defs refs in json_schema response_format (#45659)
Backport of #45659 to stable/1.104.x.
Cherry-picked from fa7ad80a62 (main).
Test-file conflict resolved by keeping the line's file plus this PR's own additions.
2026-10-20 15:32:30 -07:00
devin-ai-integration[bot]
cfdfb63a4c fix(responses): report truncated bridged output as incomplete and echo request params (#44460)
Backport of #44460 to stable/1.104.x.
Cherry-picked from f81a3f5243 (main).
2026-10-20 15:30:01 -07:00
joshua-berri
5346a40c37
Merge pull request #47454 from BerriAI/litellm_restore_token_count_input_precedence
Some checks are pending
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
LiteLLM Rust / rust-wheel (push) Waiting to run
fix(proxy): restore legacy token count input precedence
2026-10-19 17:59:47 -07:00
Joshua Valluru
e109f8977d fix(proxy): restore legacy token count input precedence 2026-10-19 17:50:09 -07:00
Mateo Wang
a63df167da
Merge pull request #47446 from BerriAI/litellm_backport_google_count_shapes_stable_1_104_x
fix(count_tokens): restore Google request compatibility on stable/1.104.x
2026-10-20 00:29:29 +00:00
mateo-berri
5e71466355 fix(count_tokens): normalize Google system parts and null tools 2026-10-19 16:53:38 -07:00
yuneng-jiang
5e4d160423
Merge pull request #47389 from BerriAI/litellm_backport_count_tokens_stable_1_104_x
fix(count_tokens): backport October 19 fixes to stable/1.104.x
2026-10-19 15:42:02 -07:00
mateo-berri
3f3fb2ac60 fix(count_tokens): omit local PDF parsing from stable backport 2026-10-19 15:20:52 -07:00
mateo-berri
704a15720a test(count_tokens): type PDF budget regression cases 2026-10-19 15:14:43 -07:00
mateo-berri
c881f7cfc1 fix(count_tokens): bound PDF parsing and preserve legacy fallback 2026-10-19 15:10:51 -07:00
mateo-berri
498095ee1b fix(count_tokens): retain legacy translated content fallback 2026-10-19 14:38:30 -07:00
mateo-berri
e9e7e32142 test(count_tokens): make strict error envelope coverage explicit 2026-10-19 14:30:57 -07:00
mateo-berri
d522e2e09b fix(count_tokens): preserve legacy provider error fallback on stable
Defer the breaking error policy from #46121. Adapt the #46777 count settings flow to attempt a labelled local estimate for provider failures across providers and configured count models.

Keep provider errors when local counting is disabled or cannot estimate the request, and test the shared behavior across HTTP error statuses.
2026-10-19 14:28:17 -07:00
mateo-berri
e19f8fd173 ci(cost map): regenerate the schema for the stable line model map 2026-10-19 14:08:16 -07:00
mateo-berri
2ddc8946d0 fix(count_tokens): complete the selected count settings prerequisites
Preserve the prompt normalization required by #46777 from #46839 and classify its token_count_tokenizer key using the upstream schema definition.
2026-10-19 14:07:15 -07:00
mateo-berri
828ddf0c37 fix(count_tokens): narrow stable backport to the October 19 patch
Keep #43900, #45301, #46121, #46784, #46759, #46767, #46772, #46815, #47057 and #46777. Port only the Gemini prerequisites from #46430.

Remove the deferred Bedrock Mantle behavior and unrelated picks from the final tree. Restore the PDF test provider fixture and deployment, and retain required test imports.
2026-10-19 14:03:42 -07:00
mateo-berri
94828be8b0 test(integration): add the openai_wire helper the backported chat template wire test imports
Hand-ported verbatim to stable/1.104.x from main.
2026-10-19 12:35:34 -07:00
Mateo Wang
c3d07e70d6 fix(count_tokens)!: count tools the way DeepSeek V4 and Qwen3.5 chat templates render them (#47173)
Backport of #47173 to stable/1.104.x.
Cherry-picked from c82e8244a5 (main).
2026-10-19 12:35:33 -07:00
Mateo Wang
b6f1a033de feat(count_tokens)!: configurable token_count_framing and token_count_model, count source in the response (#46777)
Backport of #46777 to stable/1.104.x.
Cherry-picked from 75faaeab64 (main).
2026-10-19 12:35:33 -07:00
mateo-berri
52de3b0a29 test(integration): add the support helpers the backported count_tokens wire tests import
Hand-ported to stable/1.104.x from main (bedrock_runtime_peer, provider, _cache_control_marks_support verbatim, graceful_stop_seconds at the line's 30s stop wait).
2026-10-19 12:35:32 -07:00
Mateo Wang
748e03088d fix(count_tokens)!: answer 400 for Vertex partner models without a count API and OpenAI-shaped provider errors on /v1/responses/input_tokens (#47163)
Backport of #47163 to stable/1.104.x.
Cherry-picked from ef1d3959a0 (main).
2026-10-19 12:35:32 -07:00
Mateo Wang
da5a9ffd2e ci(cost map): remove the generated model prices JSON schema (#47064)
Backport of #47064 to stable/1.104.x.
Cherry-picked from ae6f7f6613 (main).

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-19 12:35:32 -07:00
Mateo Wang
a4be1c1d13 ci(cost map): classify token_count_model as an object key in the schema generator (#47057)
Backport of #47057 to stable/1.104.x.
Cherry-picked from 6e191a58f1 (main).
2026-10-19 12:35:32 -07:00
Mateo Wang
b1037d2121 fix(proxy)!: count prompt-only token_counter requests with the provider when call_endpoint is set (#46839)
Backport of #46839 to stable/1.104.x.
Cherry-picked from 2dc6aafa6b (main).
2026-10-19 12:35:32 -07:00
Mateo Wang
4d5a6577c2 fix(proxy)!: count endpoints return errors in their source API shape instead of silent local counts (#46815)
Backport of #46815 to stable/1.104.x.
Cherry-picked from b21e659c24 (main).

Co-authored-by: Mrinal Chanshetty <mrinal@berri.ai>
2026-10-19 12:35:32 -07:00
Mateo Wang
13b4537c81 fix(count_tokens): return a 400 naming Anthropic blocks the OpenAI and Gemini count can't convert (#46772)
Backport of #46772 to stable/1.104.x.
Cherry-picked from 79ca820933 (main).

Co-authored-by: Mrinal Chanshetty <mrinal@berri.ai>
2026-10-19 12:35:32 -07:00
Mateo Wang
350def8ccf fix(count_tokens): count Anthropic-format requests for Gemini the way /v1/messages inference converts them (#46767)
Backport of #46767 to stable/1.104.x.
Cherry-picked from dba86bec28 (main).

Co-authored-by: Mrinal Chanshetty <mrinal@berri.ai>
2026-10-19 12:35:32 -07:00
Mateo Wang
531b66719a fix(count_tokens): count Anthropic-format requests for OpenAI the way /v1/messages inference converts them (#46759)
Backport of #46759 to stable/1.104.x.
Cherry-picked from d25b58f744 (main).

Co-authored-by: Mrinal Chanshetty <mrinal@berri.ai>
2026-10-19 12:35:32 -07:00
Mateo Wang
c99e4b86db ci(cost map): classify token_count_framing and token_count_fallback_model in the schema generator (#46784)
Backport of #46784 to stable/1.104.x.
Cherry-picked from c6a4437e98 (main).
2026-10-19 12:20:03 -07:00
mrinal-berri
fe9e6381a3 fix(count_tokens): count system and tools on Vertex Gemini, fall back to Mantle on Bedrock 404 (#46430)
Backport of #46430 to stable/1.104.x.
Cherry-picked from 08a24d391a (main).

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-19 12:20:03 -07:00
Krrish Dholakia
c703543b30 fix(proxy): raise on provider count_tokens errors instead of falling back to tiktoken (#46121)
Backport of #46121 to stable/1.104.x.
Cherry-picked from 719c44db3f (main).

Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
2026-10-19 12:20:03 -07:00
devin-ai-integration[bot]
095c4eaa41 fix(anthropic): count a leading system run through count_tokens' system parameter (#45463)
Backport of #45463 to stable/1.104.x.
Cherry-picked from d34ad35281 (main).

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-19 12:20:03 -07:00
devin-ai-integration[bot]
0b148c95ee fix(token_counter): price a base64 PDF document per page instead of as one image (#45301)
Backport of #45301 to stable/1.104.x.
Cherry-picked from 50ab9a3211 (main).

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-19 12:20:03 -07:00
devin-ai-integration[bot]
8222299f1d fix(bedrock): count tokens on bedrock-mantle when bedrock-runtime cannot count a Claude model (#45317)
Backport of #45317 to stable/1.104.x.
Cherry-picked from 577d74c1ce (main).
2026-10-19 12:20:03 -07:00
devin-ai-integration[bot]
3da404a7c1 fix(bedrock_mantle): add is_mantle_claude_model for the count_tokens picks
Hand-port of the is_mantle_claude_model helper from #43646 to stable/1.104.x, which #45317 imports.
Taken from 01b4ffe16b (main).
2026-10-19 11:36:30 -07:00
tin-berri
2f968f566b fix(anthropic): normalize images for provider token counting (#45185)
Backport of #45185 to stable/1.104.x.
Cherry-picked from 6befb9ad7d (main).
2026-10-19 11:36:30 -07:00
devin-ai-integration[bot]
a6a1956a17 fix(vertex_ai): forward system and tools to partner model count_tokens (#43900)
Backport of #43900 to stable/1.104.x.
Cherry-picked from 6d73fa6b49 (main).

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: jesus <jesus@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-19 11:36:30 -07:00
devin-ai-integration[bot]
9d0cca7ed0 fix(token_counter): count Gemini function_declarations tools (#43417)
Backport of #43417 to stable/1.104.x.
Cherry-picked from 8e6d99d74a (main).

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-19 11:36:30 -07:00
Mateo Wang
b40bd7bf03
Merge pull request #47344 from BerriAI/litellm_backport_47067_stable_1_104_x
fix(proxy): backport #47067 team-scoped batch model access to stable/1.104.x for v1.104.4
2026-10-19 18:29:05 +00:00