scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-10 22:41:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 83

perf: remove redundant get_llm_provider call in function_setup

Browse source 
The get_llm_provider call was used to determine if the target model
is a Gemini model for thought signature removal. This call is redundant
because _is_gemini_model already has a fallback that checks if "gemini"
is in the model name, which covers all cases where get_llm_provider
would return "gemini" as the provider.

Profiling shows this reduces function_setup time by ~37% (8.18s -> 5.19s
across 6000 requests), saving ~500µs per request.
This commit is contained in:
Ryan Crabbe 2026-01-26 16:21:00 -08:00
parent 8908eff7b1
commit 3e0d77b724
1 changed files with 0 additions and 16 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
litellm/utils.py
View file

@ -973,28 +973,12 @@ def function_setup( # noqa: PLR0915
# signatures to ensure compatibility.
if isinstance(messages, list) and len(messages) > 0:
try:
from litellm.litellm_core_utils.get_llm_provider_logic import (
get_llm_provider,
)
from litellm.litellm_core_utils.prompt_templates.factory import (
THOUGHT_SIGNATURE_SEPARATOR,
)
# Get custom_llm_provider to determine target provider
custom_llm_provider = kwargs.get("custom_llm_provider")
# If custom_llm_provider not in kwargs, try to determine it from the model
if not custom_llm_provider and model:
try:
_, custom_llm_provider, _, _ = get_llm_provider(
model=model,
custom_llm_provider=custom_llm_provider,
)
except Exception:
# If we can't determine the provider, skip this processing
pass
# Only process if target is NOT a Gemini model
if not _is_gemini_model(model, custom_llm_provider):
verbose_logger.debug(
"Removing thought signatures from tool call IDs for non-Gemini model"