mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-27 06:13:07 +00:00
55c54bb3c0
The gateway now authenticates LiteLLM virtual keys as well as the master key. Keys are looked up by sha256 hash in the proxy's LiteLLM_VerificationToken table (general_settings.database_url or DATABASE_URL), honoring expires, blocked, models and allowed_routes, and are read from Authorization: Bearer, x-api-key or api-key The same query LEFT JOINs the key's team. A blocked or missing team refuses the key and the team's models narrow the key's, as in the proxy's can_key_call_model and can_team_access_model. The key's team also stays in its grant: a deployment whose model_info.team_id names a team serves only that team's keys, so routing and retries skip it for any other virtual key, as the proxy's filter_team_based_models does. The master key routes as before Keys with budgets pass without spend tracking: the gateway does not record spend for its own requests, so a key or team gets the proxy's budget-exceeded error only once the spend the Python proxy has stored reaches its max_budget Every other restriction the proxy enforces and this gateway does not yet fails closed. The query also joins the key's team membership, user and budget tables and reduces each restriction column to whether it is set, and a key carrying one (rate limits and max_parallel_requests, including those in a budget table or per model in model_max_budget, a membership's allowed_models, projects, organizations, agents, guardrails, tags, tool allowlists, enforced params and the like, on the key, its team, its membership or its user) gets a 401 saying that kind of key is not supported by this gateway yet Lookups sit behind a moka cache (60 s for found keys, 5 s for unknown ones) with single-flight misses. When the database fails, recently seen keys keep working from a stale copy and store reads pause briefly; the master key never needs the database. Auth failures return the proxy's ProxyException JSON, including 403 key_model_access_denied for a model outside the key's list The load-test README documents running with virtual keys and x-api-key, and key-churn asks for the 4 model groups genconfig creates by default |
||
|---|---|---|
| .. | ||
| build-images.sh | ||
| cloud-run.sh | ||