mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-27 01:22:18 +00:00
f22eeb2ce0
`ProxyConfig.get_config()` walked the parsed config and replaced every `os.environ/<KEY>` string with `get_secret(value)` before anything initialized the secret manager, so a key held only by the manager resolved to `None` and that `None` was written back into the config. The later fallback in `load_config` could not recover it, because the key now existed with a `None` value. Hoist the initialization into `get_config()`, ahead of the resolution pass, so every entrypoint gets it: the CLI already did this itself, but the microservice entrypoints (`gateway/main.py`, `backend/main.py`) uvicorn the app directly and bypass the CLI. `load_config`'s own call is now redundant and is dropped, so startup builds the manager once instead of building one and discarding it. `get_config()` also runs on management-endpoint request paths, so this returns early once a manager exists rather than rebuilding the client per request. Also warn when a reference the manager would have been asked for resolves to `None`. The reporter had no log line at all to work from. `get_secret` only reaches the manager when reads are enabled and the name is in `hosted_keys`, so `secret_manager_would_be_consulted` mirrors that gate and keeps the warning off env-only references, which are expected rather than an error.
29 lines
1.3 KiB
Python
29 lines
1.3 KiB
Python
"""Resolve a proxy config's ``model_list`` for the Rust AI gateway.
|
|
|
|
The Rust gateway calls this once at load time (via an embedded interpreter) and
|
|
builds its own (Rust) router from the returned ``model_list``. We do NOT call
|
|
``ProxyConfig.load_config`` here: that returns a *Python* ``litellm.Router`` (not
|
|
usable from Rust) and boots the whole proxy (callbacks, cache, DB, auth) as side
|
|
effects.
|
|
|
|
Instead we reuse ``ProxyConfig.get_config`` — the actual config reader — so the
|
|
gateway inherits the same heavy lifting the proxy does: ``include:`` merging,
|
|
``os.environ/`` + secret-manager resolution, and DB-stored models (when a DB is
|
|
configured). Its only proxy-setup side effect is bringing up the configured
|
|
secret manager, which is what makes that resolution work. Returns the resolved
|
|
``model_list``; the Rust side deserializes each entry into its ``Deployment``.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
from typing import Any, Final
|
|
|
|
|
|
def read_model_list(config_path: str) -> list[dict[str, Any]]:
|
|
"""Load ``config_path`` via the proxy's own reader and return its
|
|
resolved ``model_list``."""
|
|
from litellm.proxy.proxy_server import ProxyConfig
|
|
|
|
config: Final = asyncio.run(ProxyConfig().get_config(config_file_path=config_path))
|
|
return config.get("model_list") or []
|