scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-21 00:21:49 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 73

fix(health-check): strip max_tokens before non-chat handlers

Browse source 
`_update_litellm_params_for_health_check` unconditionally injects
`messages` and `max_tokens` (resolved via
`_resolve_health_check_max_tokens` /
`BACKGROUND_HEALTH_CHECK_MAX_TOKENS`) into the litellm params for every
deployment. `_filter_model_params` was only stripping `messages`, so
`max_tokens` reached every non-chat handler in
`HealthCheckHelpers.get_mode_handlers` — image/video generation,
embedding, rerank, audio_speech, audio_transcription, ocr, responses,
batch.

OpenAI's image-generation endpoints (dall-e-2, dall-e-3, gpt-image-1)
are strict and return 400 `Unknown parameter: 'max_tokens'`, so any
deployment with `mode: image_generation` is permanently reported
unhealthy in the proxy UI. Manually calling
`litellm.aimage_generation(...)` works fine; only the automated
health check fails. Other providers happen to silently drop unknown
fields, hiding the issue.

Strip both `messages` and `max_tokens` in `_filter_model_params`.
`acompletion` is the only mode handler that consumes `model_params`
unfiltered, so this leaves chat health checks unaffected. The other
handlers don't accept `max_tokens` to begin with.
This commit is contained in:
xianren 2026-04-27 22:01:27 +08:00
parent 0644a1b02b
commit cfe0b774ae
1 changed files with 21 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

23
litellm/litellm_core_utils/health_check_utils.py
View file

@ -2,10 +2,29 @@
Utils used for litellm.ahealth_check()
"""
#: litellm params that ``_update_litellm_params_for_health_check`` injects for
#: chat-completion health checks but that are invalid (or rejected) on every
#: other handler in :class:`HealthCheckHelpers.get_mode_handlers` — image and
#: video generation, embeddings, audio speech / transcription, rerank, ocr,
#: responses, batch, etc. ``messages`` is always chat-only; ``max_tokens`` is
#: chat/completion-only and is rejected by strict providers (e.g. OpenAI's
#: image-generation endpoints return 400 ``Unknown parameter: 'max_tokens'``).
_NON_CHAT_HEALTH_CHECK_STRIP_KEYS = {"messages", "max_tokens"}
def _filter_model_params(model_params: dict) -> dict:
"""Remove 'messages' param from model params."""
return {k: v for k, v in model_params.items() if k != "messages"}
"""Strip chat-only params before invoking a non-chat health check handler.
``litellm.acompletion`` is the only mode handler that consumes
``model_params`` unfiltered; every other handler routes through this
helper, so removing chat-completion-only keys here keeps strict providers
(OpenAI image generation, etc.) from rejecting the request.
"""
return {
k: v
for k, v in model_params.items()
if k not in _NON_CHAT_HEALTH_CHECK_STRIP_KEYS
}
def _create_health_check_response(response_headers: dict) -> dict: