scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-12 23:01:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 88

fix(otel): keep the generic callback out of the model-prefix namespace

Browse source 
The router treats every entry in _known_custom_logger_compatible_callbacks as a
prompt-management provider when it appears before the / of a model string, so
adding "generic" as a callback name silently reserved the generic/ prefix
product-wide: a pre-existing deployment with model: generic/<x> stopped routing
and answered 500 'Prompt ID is not set', with the OTEL v2 flag off.

Split the two uses of that list. The callback list keeps "generic" so
callbacks: ["generic"] still initializes and exports; the router's prefix test
reads a separate set that excludes callback names which are ordinary words.

Verified base-vs-head on a live proxy with the flag unset: generic/<x> now
answers 400 'no healthy deployments', byte-identical to base, while
callbacks: ["generic"] still delivers the gen-AI span to its exporter.
This commit is contained in:
Yucheng Zhu 2026-08-12 22:52:47 -07:00
parent 29806dee16
commit 13b518830c
2 changed files with 10 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
litellm/__init__.py
View file

@ -167,6 +167,14 @@ _custom_logger_compatible_callbacks_literal = Literal[
cold_storage_custom_logger: Optional[_custom_logger_compatible_callbacks_literal] = None
logged_real_time_event_types: Optional[Union[List[str], Literal["*"]]] = None
_known_custom_logger_compatible_callbacks: List = list(get_args(_custom_logger_compatible_callbacks_literal))
# Callback names that must never be read as a model-name prefix. The router treats any
# entry in the callback list as a prompt-management provider when it appears before the
# "/" of a model string, so a callback named for an ordinary word would silently reserve
# that prefix product-wide and break existing deployments that already use it.
_CALLBACKS_NOT_USABLE_AS_MODEL_PREFIX: Final = frozenset({"generic"})
_known_prompt_management_prefixes: Final = frozenset(
name for name in _known_custom_logger_compatible_callbacks if name not in _CALLBACKS_NOT_USABLE_AS_MODEL_PREFIX
)
callbacks: List[
Union[Callable, _custom_logger_compatible_callbacks_literal, "CustomLogger"] # CustomLogger is lazy-loaded
] = []

4
litellm/router.py
View file

@ -3734,7 +3734,7 @@ class Router:
return False
split_litellm_model: Final = litellm_model.split("/")[0]
return split_litellm_model in litellm._known_custom_logger_compatible_callbacks
return split_litellm_model in litellm._known_prompt_management_prefixes
async def _prompt_management_factory(
self,
@ -8181,7 +8181,7 @@ class Router:
if "/" in litellm_model:
split_litellm_model: Final = litellm_model.split("/")[0]
if split_litellm_model in litellm._known_custom_logger_compatible_callbacks:
if split_litellm_model in litellm._known_prompt_management_prefixes:
is_prompt_management_model = True
if is_prompt_management_model: