scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-05 02:41:56 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 48

perf: cache _get_relevant_args_to_use_for_logging() as module-level frozenset

Browse source 
The set of valid LLM API parameter names for logging was being rebuilt
on every request from 8 OpenAI SDK type annotations + set operations.
Since these are static TypedDict annotations that never change at
runtime, compute once at import time and store as a class-level
frozenset.

Line profiler: get_standard_logging_model_parameters() dropped from
774ms to 77ms across 12K calls (90% reduction, ~25µs/req saved).
This commit is contained in:
Ryan Crabbe 2026-01-30 10:06:20 -08:00 • committed by Alexsander Hamir
parent cbc366f0d7
commit a06359d40a
1 changed files with 9 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
litellm/litellm_core_utils/model_param_helper.py
View file

@ -17,15 +17,16 @@ from litellm.types.rerank import RerankRequest
class ModelParamHelper:
# Cached at class level — deterministic set built from static OpenAI type annotations
_relevant_logging_args: frozenset = frozenset()
@staticmethod
def get_standard_logging_model_parameters(
model_parameters: dict,
) -> dict:
""" """
standard_logging_model_parameters: dict = {}
supported_model_parameters = (
ModelParamHelper._get_relevant_args_to_use_for_logging()
)
supported_model_parameters = ModelParamHelper._relevant_logging_args
for key, value in model_parameters.items():
if key in supported_model_parameters:
@ -172,3 +173,8 @@ class ModelParamHelper:
Get the kwargs to exclude from the cache key
"""
return set(["metadata"])
ModelParamHelper._relevant_logging_args = frozenset(
ModelParamHelper._get_relevant_args_to_use_for_logging()
)