scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-22 00:31:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 96

perf: skip Pydantic model construction in get_api_base when api_base is in dict

Browse source 
Add fast path to check optional_params.get("api_base") directly before
constructing a full LiteLLM_Params Pydantic model. When api_base is present
(the common case via router), return it immediately — avoiding ~29µs of
Pydantic validation overhead per request.

Profiled: get_api_base 30.5µs/call → 1.2µs/call (-96%)
This commit is contained in:
Ryan Crabbe 2026-01-28 17:30:41 -08:00
parent d12ce3cd5d
commit f655f47d84
1 changed files with 7 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
litellm/litellm_core_utils/llm_response_utils/get_api_base.py
View file

@ -28,6 +28,13 @@ def get_api_base(
```
"""
# Fast path: if api_base is already in the dict, return it directly
# without constructing a full LiteLLM_Params Pydantic model.
if isinstance(optional_params, dict):
_api_base = optional_params.get("api_base")
if _api_base is not None:
return _api_base
try:
if isinstance(optional_params, LiteLLM_Params):
_optional_params = optional_params