fix: get_model_info asks Hugging Face for the repo id, not the routing prefix

`_get_model_info_helper` has `split_model` -- the bare repo id -- and passed
`model` to `_get_max_position_embeddings` instead, so the address requested was
`huggingface.co/huggingface/<org>/<model>`. That is not a repo: it answers 404,
and `get_model_info` returns `max_tokens=None` for every Hugging Face model.

`get_max_tokens` does the same branch correctly one function away, by
reassigning `model` from `get_llm_provider` before the check.

Measured on 1.85.0, and unchanged on main:

    get_model_info("huggingface/deepseek-ai/DeepSeek-V4-Pro")
      -> {} after 533ms, GET .../huggingface/deepseek-ai/DeepSeek-V4-Pro/raw/main/config.json (404)
    get_model_info("deepseek-ai/DeepSeek-V4-Pro")
      -> BadRequestError in 1ms (no provider), so a caller cannot work around it

The existing `hf_config_route` fixture answers any path ending in config.json,
so the returned value cannot catch this. The new test reads the requested URL,
and fails on the previous code with the doubled segment.
This commit is contained in:
zhanghui 2026-09-20 11:47:00 +08:00
parent 4011367b39
commit 5fe7e3cd7c
2 changed files with 24 additions and 1 deletions

View file

@ -5814,7 +5814,7 @@ def _get_model_info_helper(
)
if custom_llm_provider == "huggingface":
max_tokens: Final = _get_max_position_embeddings(model_name=model)
max_tokens: Final = _get_max_position_embeddings(model_name=split_model)
return ModelInfoBase(
key=model,
max_tokens=max_tokens,

View file

@ -5238,6 +5238,29 @@ class TestHuggingFaceConfigFetch:
request_timeout = hf_config_route.calls.last.request.extensions["timeout"]
assert request_timeout["read"] == HF_CONFIG_FETCH_TIMEOUT_SECONDS
def test_get_model_info_asks_for_the_repo_path_without_the_routing_prefix(self, hf_config_route):
"""The repo id is the path; the `huggingface/` routing prefix is not part of it.
`_get_model_info_helper` has `split_model` -- the bare repo id -- and used to
hand `model` to the fetch instead, so the address asked for was
`huggingface.co/huggingface/<org>/<model>`, which is not a repo and answers 404.
The window came back None for every Hugging Face model. `get_max_tokens` one
function away does the same branch correctly, by reassigning `model` from
`get_llm_provider` first.
The mock answers any path ending in config.json, so the returned value alone
cannot catch this -- the URL is what the assertion has to read.
"""
from litellm.utils import get_model_info
info = get_model_info("huggingface/some-org/some-model")
assert info["max_tokens"] == 512
assert (
str(hf_config_route.calls.last.request.url)
== "https://huggingface.co/some-org/some-model/raw/main/config.json"
)
def test_get_max_position_embeddings_reads_hf_config_with_a_bounded_timeout(self, hf_config_route):
from litellm.constants import HF_CONFIG_FETCH_TIMEOUT_SECONDS
from litellm.utils import _get_max_position_embeddings