Commit graph

19664 commits

Author SHA1 Message Date
joshua-berri
f5049dba86 fix(bedrock): resolve profile pricing before applying custom rates (#47419)
Backport of #47419 to stable/1.104.x.
Cherry-picked from 3124797399 (main).
Ported onto the line's register_model and router startup path without #41523's aliased-entry rewrite: registered fields come from the side table added in the previous commit. Tests adapted to the line: private names (_replay_model_cost_registrations, _response_cost_calculator), the custom-pricing test helpers from main's test_cost_calculator.py, CONVERSE_ANSWER inlined, and the alias-propagation and bedrock gpt-6.1-sol/ARN cases removed since those depend on #41523 and #46128, which are not on this line.
2026-10-20 17:21:34 -07:00
joshua-berri
21a84f9e8f fix(batch): preserve recoverable costs and prevent poll starvation (#47190)
Backport of #47190 to stable/1.104.x.
Cherry-picked from 0ede0e10e7 (main).
Ported by hand onto the line's poller (the ManagedObjectRow protocol is the line's local _ManagedObjectRow; _finalize_unbilled_terminal_job keeps the line's None return). Test conflicts take this PR's assertions; the line-item poller tests from main are not on this line.
2026-10-20 17:21:34 -07:00
Kerry Lu
5684267b12 fix(logging): report the cost-map entry billing used in model_map_information (#47182)
Backport of #47182 to stable/1.104.x.
Cherry-picked from 1212260ab1 (main).
Adapted to the line's private helper names (_select_model_name_for_cost_calc, _response_cost_calculator); the new test is appended to the line's test file.
2026-10-20 16:12:01 -07:00
Paul El Khoury
54f9259b40 fix(openai): drop unsupported reasoning effort for gpt-6.1-sol (#43935)
Backport of #43935 to stable/1.104.x.
Cherry-picked from d7620c77a1 (main).
Adapted to the line: the bridge check lives in litellm/main.py rather than litellm/responses/bridge_check.py, the effort helpers keep the line's private names, and the azure gpt-6.1-sol rows get supports_none_reasoning_effort=false (from #46377 on main) since this fix and its tests key on it. Test files keep the line's content plus this PR's own hunks.
2026-10-20 16:12:01 -07:00
joshua-berri
f7b0b09e75 fix(batches): preserve spend across malformed output and metadata failures (#47184)
Backport of #47184 to stable/1.104.x.
Cherry-picked from 3fb95c8fe3 (main).
Ported by hand onto the line's poller, whose surrounding code predates several main-only batch features: the claim now writes the terminal status and NUL-stripped file_object, a failed spend-log write restores both, and the managed-files postgres_storable_json sanitizer (from #45703 on main) is added to the line since this fix needs it.
2026-10-20 16:12:01 -07:00
joshua-berri
bb1e40c3e5 fix(batches): handle empty models when logging batch cost (#47155)
Backport of #47155 to stable/1.104.x.
Cherry-picked from 4143e0869e (main).
2026-10-20 16:12:01 -07:00
devin-ai-integration[bot]
2ac8b35a05 fix(model_prices): consolidate claude-haiku-5-5 over-100k pricing and capability flags (#45151)
Backport of #45151 to stable/1.104.x.
Cherry-picked from 46d440ae96 (main).
Conflicts resolved by keeping the line's fields and adding only the over-100k fields this PR introduces; cost-map entries replaced per model.
2026-10-20 16:12:00 -07:00
Krrish Dholakia
d3366b93b4 feat(model_prices): add claude-haiku-5-5 model pricing (#45108)
Backport of #45108 to stable/1.104.x.
Cherry-picked from 5cd1126870 (main).
Prerequisite for #45151: adds the claude-haiku-5-5 entries that fix corrects. Cost-map conflicts resolved by appending the new entries.
2026-10-20 16:12:00 -07:00
devin-ai-integration[bot]
61c5d953ee fix(anthropic): add the dangerous-tool-use beta when a /v1/messages request carries safeguards (#43955)
Backport of #43955 to stable/1.104.x.
Cherry-picked from eb5295b20c (main).
Adapted to the line: the transformation lives under anthropic/experimental_pass_through, the integration tests live under tests/integration/providers, and the CLI-beta wire assertion change is dropped since that test is not on the line.
2026-10-20 16:12:00 -07:00
mrinal-berri
2f19d8ed87 fix(databricks): send a single plain text block as a string for non-Claude models (#45686)
Backport of #45686 to stable/1.104.x.
Cherry-picked from 1aee1a21ed (main).
Test-file import conflict resolved by keeping the line's imports plus this PR's own.
2026-10-20 15:32:54 -07:00
mrinal-berri
6ba8151a2d fix(databricks): keep #/$defs refs in json_schema response_format (#45659)
Backport of #45659 to stable/1.104.x.
Cherry-picked from fa7ad80a62 (main).
Test-file conflict resolved by keeping the line's file plus this PR's own additions.
2026-10-20 15:32:30 -07:00
devin-ai-integration[bot]
cfdfb63a4c fix(responses): report truncated bridged output as incomplete and echo request params (#44460)
Backport of #44460 to stable/1.104.x.
Cherry-picked from f81a3f5243 (main).
2026-10-20 15:30:01 -07:00
Joshua Valluru
e109f8977d fix(proxy): restore legacy token count input precedence 2026-10-19 17:50:09 -07:00
mateo-berri
5e71466355 fix(count_tokens): normalize Google system parts and null tools 2026-10-19 16:53:38 -07:00
mateo-berri
3f3fb2ac60 fix(count_tokens): omit local PDF parsing from stable backport 2026-10-19 15:20:52 -07:00
mateo-berri
704a15720a test(count_tokens): type PDF budget regression cases 2026-10-19 15:14:43 -07:00
mateo-berri
c881f7cfc1 fix(count_tokens): bound PDF parsing and preserve legacy fallback 2026-10-19 15:10:51 -07:00
mateo-berri
498095ee1b fix(count_tokens): retain legacy translated content fallback 2026-10-19 14:38:30 -07:00
mateo-berri
e9e7e32142 test(count_tokens): make strict error envelope coverage explicit 2026-10-19 14:30:57 -07:00
mateo-berri
d522e2e09b fix(count_tokens): preserve legacy provider error fallback on stable
Defer the breaking error policy from #46121. Adapt the #46777 count settings flow to attempt a labelled local estimate for provider failures across providers and configured count models.

Keep provider errors when local counting is disabled or cannot estimate the request, and test the shared behavior across HTTP error statuses.
2026-10-19 14:28:17 -07:00
mateo-berri
828ddf0c37 fix(count_tokens): narrow stable backport to the October 19 patch
Keep #43900, #45301, #46121, #46784, #46759, #46767, #46772, #46815, #47057 and #46777. Port only the Gemini prerequisites from #46430.

Remove the deferred Bedrock Mantle behavior and unrelated picks from the final tree. Restore the PDF test provider fixture and deployment, and retain required test imports.
2026-10-19 14:03:42 -07:00
mateo-berri
94828be8b0 test(integration): add the openai_wire helper the backported chat template wire test imports
Hand-ported verbatim to stable/1.104.x from main.
2026-10-19 12:35:34 -07:00
Mateo Wang
c3d07e70d6 fix(count_tokens)!: count tools the way DeepSeek V4 and Qwen3.5 chat templates render them (#47173)
Backport of #47173 to stable/1.104.x.
Cherry-picked from c82e8244a5 (main).
2026-10-19 12:35:33 -07:00
Mateo Wang
b6f1a033de feat(count_tokens)!: configurable token_count_framing and token_count_model, count source in the response (#46777)
Backport of #46777 to stable/1.104.x.
Cherry-picked from 75faaeab64 (main).
2026-10-19 12:35:33 -07:00
mateo-berri
52de3b0a29 test(integration): add the support helpers the backported count_tokens wire tests import
Hand-ported to stable/1.104.x from main (bedrock_runtime_peer, provider, _cache_control_marks_support verbatim, graceful_stop_seconds at the line's 30s stop wait).
2026-10-19 12:35:32 -07:00
Mateo Wang
748e03088d fix(count_tokens)!: answer 400 for Vertex partner models without a count API and OpenAI-shaped provider errors on /v1/responses/input_tokens (#47163)
Backport of #47163 to stable/1.104.x.
Cherry-picked from ef1d3959a0 (main).
2026-10-19 12:35:32 -07:00
Mateo Wang
da5a9ffd2e ci(cost map): remove the generated model prices JSON schema (#47064)
Backport of #47064 to stable/1.104.x.
Cherry-picked from ae6f7f6613 (main).

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-19 12:35:32 -07:00
Mateo Wang
a4be1c1d13 ci(cost map): classify token_count_model as an object key in the schema generator (#47057)
Backport of #47057 to stable/1.104.x.
Cherry-picked from 6e191a58f1 (main).
2026-10-19 12:35:32 -07:00
Mateo Wang
b1037d2121 fix(proxy)!: count prompt-only token_counter requests with the provider when call_endpoint is set (#46839)
Backport of #46839 to stable/1.104.x.
Cherry-picked from 2dc6aafa6b (main).
2026-10-19 12:35:32 -07:00
Mateo Wang
4d5a6577c2 fix(proxy)!: count endpoints return errors in their source API shape instead of silent local counts (#46815)
Backport of #46815 to stable/1.104.x.
Cherry-picked from b21e659c24 (main).

Co-authored-by: Mrinal Chanshetty <mrinal@berri.ai>
2026-10-19 12:35:32 -07:00
Mateo Wang
13b4537c81 fix(count_tokens): return a 400 naming Anthropic blocks the OpenAI and Gemini count can't convert (#46772)
Backport of #46772 to stable/1.104.x.
Cherry-picked from 79ca820933 (main).

Co-authored-by: Mrinal Chanshetty <mrinal@berri.ai>
2026-10-19 12:35:32 -07:00
Mateo Wang
350def8ccf fix(count_tokens): count Anthropic-format requests for Gemini the way /v1/messages inference converts them (#46767)
Backport of #46767 to stable/1.104.x.
Cherry-picked from dba86bec28 (main).

Co-authored-by: Mrinal Chanshetty <mrinal@berri.ai>
2026-10-19 12:35:32 -07:00
Mateo Wang
531b66719a fix(count_tokens): count Anthropic-format requests for OpenAI the way /v1/messages inference converts them (#46759)
Backport of #46759 to stable/1.104.x.
Cherry-picked from d25b58f744 (main).

Co-authored-by: Mrinal Chanshetty <mrinal@berri.ai>
2026-10-19 12:35:32 -07:00
Mateo Wang
c99e4b86db ci(cost map): classify token_count_framing and token_count_fallback_model in the schema generator (#46784)
Backport of #46784 to stable/1.104.x.
Cherry-picked from c6a4437e98 (main).
2026-10-19 12:20:03 -07:00
mrinal-berri
fe9e6381a3 fix(count_tokens): count system and tools on Vertex Gemini, fall back to Mantle on Bedrock 404 (#46430)
Backport of #46430 to stable/1.104.x.
Cherry-picked from 08a24d391a (main).

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-19 12:20:03 -07:00
Krrish Dholakia
c703543b30 fix(proxy): raise on provider count_tokens errors instead of falling back to tiktoken (#46121)
Backport of #46121 to stable/1.104.x.
Cherry-picked from 719c44db3f (main).

Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
2026-10-19 12:20:03 -07:00
devin-ai-integration[bot]
095c4eaa41 fix(anthropic): count a leading system run through count_tokens' system parameter (#45463)
Backport of #45463 to stable/1.104.x.
Cherry-picked from d34ad35281 (main).

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-19 12:20:03 -07:00
devin-ai-integration[bot]
0b148c95ee fix(token_counter): price a base64 PDF document per page instead of as one image (#45301)
Backport of #45301 to stable/1.104.x.
Cherry-picked from 50ab9a3211 (main).

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-19 12:20:03 -07:00
devin-ai-integration[bot]
8222299f1d fix(bedrock): count tokens on bedrock-mantle when bedrock-runtime cannot count a Claude model (#45317)
Backport of #45317 to stable/1.104.x.
Cherry-picked from 577d74c1ce (main).
2026-10-19 12:20:03 -07:00
tin-berri
2f968f566b fix(anthropic): normalize images for provider token counting (#45185)
Backport of #45185 to stable/1.104.x.
Cherry-picked from 6befb9ad7d (main).
2026-10-19 11:36:30 -07:00
devin-ai-integration[bot]
a6a1956a17 fix(vertex_ai): forward system and tools to partner model count_tokens (#43900)
Backport of #43900 to stable/1.104.x.
Cherry-picked from 6d73fa6b49 (main).

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: jesus <jesus@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-19 11:36:30 -07:00
devin-ai-integration[bot]
9d0cca7ed0 fix(token_counter): count Gemini function_declarations tools (#43417)
Backport of #43417 to stable/1.104.x.
Cherry-picked from 8e6d99d74a (main).

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-19 11:36:30 -07:00
mateo-berri
c47b760253 test(proxy): import Final in the line's batch endpoint tests
The tests picked from #47067 annotate module constants with Final, which main's copy imports and this line's did not, so the module failed to import on Python 3.12.
2026-10-19 09:10:10 -07:00
Mrunmayee Daware
e87e661118 fix(proxy): authorize the public name of a team-scoped model on batch retrieve/cancel (#47067)
Backport of #47067 to stable/1.104.x.
Cherry-picked from e979bd0ecf (main).
Conflict: main renamed the model access helper to check_model_access_helper, while stable/1.104.x still names it _check_model_access_helper, so the new owning-team check calls it by the line's name.
Line adaptation: the line's test_auth_utils.py had no module-level Router import, which main's copy has and the picked helper uses, so it is added.
2026-10-19 08:57:08 -07:00
Yuneng Jiang
8a834c071e
fix(guardrails): import the decisions provider list from litellm.decisions.call on stable/1.104.x
The decision model guardrail backport (#45900) added supported_decisions_providers to
litellm.decisions.main and imported it from there in get_guardrail_ui_settings. The
decisions backport (#45906) then picked #45130, which rewrites litellm.decisions.main
and moves the provider list into litellm.decisions.call as _supported_providers, so the
import broke and GET /guardrails/ui/add_guardrail_settings raises ImportError. The Admin
UI's Add Provider Guardrail wizard gets no providers as a result

Match main by naming it supported_decisions_providers in litellm.decisions.call and
pointing the endpoint and its test there, the same fix rc/1.105.0 got in fcbb02231b
2026-10-11 01:16:36 -07:00
Joshua Valluru
58746028d4 test(router): verify deployed classifier cost and provider accounting 2026-10-10 19:55:21 -07:00
Joshua Valluru
dc20b834d7 feat(ui): backport deployed decision router setup to 1.104 2026-10-10 19:05:45 -07:00
mateo
c332420e7a test(unit): flush queued logging tasks after each test so systemone callback checks stay isolated
Ports the pytest_runtest_call hook from tests/unit/conftest.py on main (#44146). Without it the asystemone call type test picks up adecisions callbacks queued by earlier tests in the same worker

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-11 01:03:12 +00:00
mateo
c4822754bf chore(backport): regenerate dashboard API types and adapt the picks to rc/1.105.0
schema.d.ts is regenerated with npm run gen:api instead of hand-merged, the Strands logo the Add Model list imports is copied from main, and the picked tests get the import order and Final import this line's ruff asks for

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-11 01:03:12 +00:00
devin-ai-integration[bot]
59c675c51c feat(sdk): add litellm.systemone and trim litellm.decisions to the OpenAI shape (#45846)
* feat(sdk): add litellm.systemone and trim litellm.decisions to the OpenAI shape

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: run tests/unit/systemone in the core-utils unit shard

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(sdk): keep litellm.decisions(state=...) working with a DeprecationWarning

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: make tests/unit/systemone a package

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(sdk): surface the litellm.decisions(state=...) deprecation as a UserWarning

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 25d9b05386)
2026-10-11 01:02:56 +00:00