litellm/litellm/proxy
ryan-crabbe-berri 2792887e47
fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851)
* fix(proxy): give proxy_admin_viewer read parity with proxy_admin

Route-level checks already default-allow management GETs for the viewer
role, but ~15 handlers compared user_role to PROXY_ADMIN only, dropping
viewers into regular-user scoping (/key/list, /user/info, /model/info,
guardrails, prompts, agents, memory, workflows, MCP catalog, coordination
redis settings, credential migration check, enterprise projects). Swap
those read paths to user_api_key_has_admin_view; write gates unchanged.

The dashboard now presents the viewer session as Admin for all gating
(effectiveSessionRole) so every page fetches with admin visibility, with
userRoleLabel/isViewOnly preserving the account-menu label and the
playground cost guard. The server remains the write authority.

* refactor(agents): remove side-effectful health_check param from GET /v1/agents

Addresses a security review finding on the admin viewer read parity change:
listing agents with health_check=true made the proxy issue a server-side GET
to every agent URL, so a read-scoped caller could trigger request fan-out
beyond their object permissions. The list endpoint is now a pure read for
every role.

Removes the query param, the URL probing helper and its timeouts, the
AgentHealthCheck httpx provider tag, and the dashboard's Health Check
toggle. Requests still passing health_check=true get the full list back
with the param ignored.

* fix(proxy): keep credential encryption check proxy_admin only

The residual scan behind GET /credentials/migrate-encryption/check loads
every model, credential, MCP, team, and verification-token row and runs a
decryption attempt on each stored value. Extending it to proxy_admin_viewer
let a read-only account repeatedly trigger deployment-wide scans, so the
route keeps its original full-admin gate.

* fix(agents): restore health_check, keep list fast path proxy_admin only

Restores the agent health_check feature exactly as before this PR: the
query param, the URL probing helper, the httpx provider tag, and the
dashboard toggle all return, so existing callers keep the filtering
contract. The viewer expansion is instead reverted at its source: the
GET /v1/agents admin fast path stays PROXY_ADMIN only, so a
proxy_admin_viewer goes through the object-permission scoped branch as
before and cannot fan out health checks beyond their allowlist. The
viewer read of a single agent stays viewer-inclusive since it has no
side effects.
2026-08-05 18:33:55 +00:00
..
_experimental fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851) 2026-08-05 18:33:55 +00:00
a2a feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
agent_endpoints fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851) 2026-08-05 18:33:55 +00:00
analytics_endpoints feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
anthropic_endpoints fix(claude-code): create-only skill registration with a PUT update route (LIT-4110) (#31752) 2026-08-05 01:44:56 +00:00
auth fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851) 2026-08-05 18:33:55 +00:00
batches_endpoints chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
client feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
common_utils chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
config_management_endpoints feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
config_resolvers feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
container_endpoints chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
credential_endpoints feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
custom_hooks feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
db chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
discovery_endpoints feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
enterprise_billing feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
example_config_yaml fix(lint): bring basedpyright rule counts back under their budget limits 2026-08-05 10:23:02 -07:00
fine_tuning_endpoints feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
google_endpoints feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
guardrails fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851) 2026-08-05 18:33:55 +00:00
health_check_utils feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
health_endpoints chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
hooks chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
image_endpoints feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
logging_endpoints feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
management_endpoints fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851) 2026-08-05 18:33:55 +00:00
management_helpers chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
memory fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851) 2026-08-05 18:33:55 +00:00
middleware chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
ocr_endpoints feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
openai_evals_endpoints feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
openai_files_endpoints chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
pass_through_endpoints chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
policy_engine chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
prompts fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851) 2026-08-05 18:33:55 +00:00
public_endpoints chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
rag_endpoints feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
realtime_endpoints chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
rerank_endpoints feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
response_api_endpoints chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
response_polling feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
search_endpoints chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
shutdown feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
spend_tracking chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
swagger
test_prompts
types_utils chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
ui_crud_endpoints chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
vector_store_endpoints feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
vector_store_files_endpoints chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
vertex_ai_endpoints chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
video_endpoints feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
workflows feat(proxy): durable agent workflow run tracking via /v1/workflows/runs (#26793) 2026-04-29 17:12:18 -07:00
.gitignore
__init__.py
_lazy_features.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
_lazy_openapi_snapshot.json fix(claude-code): create-only skill registration with a PUT update route (LIT-4110) (#31752) 2026-08-05 01:44:56 +00:00
_lazy_openapi_snapshot.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
_logging.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
_new_new_secret_config.yaml Revert "chore: remove _experimental/out (#31546)" 2026-07-01 13:25:47 -07:00
_new_secret_config.yaml Revert "chore: remove _experimental/out (#31546)" 2026-07-01 13:25:47 -07:00
_super_secret_config.yaml Revert "chore: remove _experimental/out (#31546)" 2026-07-01 13:25:47 -07:00
_types.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
cached_logo.jpg fix: prompt registry 2026-02-18 00:34:54 +05:30
caching_routes.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
common_request_processing.py fix(router): redact fallback tracebacks at the call site and cover the sync deferred stream (#35843) 2026-08-05 10:06:52 -07:00
compliance_checks.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
custom_auth_auto.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
custom_prompt_management.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
custom_sso.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
custom_validate.py
dd_span_tagger.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
dev_config.yaml Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
enterprise
health_check.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
lambda.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
litellm_pre_call_utils.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
llamaguard_prompt.txt
logo.jpg
mcp_registry.json fix(mcp): use official Google Drive streamable HTTP MCP server (#34322) 2026-07-22 23:41:51 +00:00
mcp_tools.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
model_config.yaml
openapi.json
openapi_registry.json fix(mcp): add Google Sheets, Drive, Calendar, and Docs to the OpenAPI registry 2026-07-20 19:02:00 -07:00
plugin_routes.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
post_call_rules.py ci(lint): add blanket-noqa, dataclass-default, and unused-noqa Ruff rules (#30516) 2026-06-16 12:07:22 -07:00
prisma_migration.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
prometheus_cleanup.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
proxy_cli.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
proxy_config.yaml fix: prompt registry 2026-02-18 00:34:54 +05:30
proxy_server.py fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851) 2026-08-05 18:33:55 +00:00
read_model_list.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
README.md build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
route_llm_request.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
schema.prisma fix(proxy): persist periodic reload schedule state so status survives restarts and fires without store_model_in_db (#35165) 2026-08-04 15:42:57 -07:00
start.sh
utils.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
wildcard_config.yaml feat: add dev and wildcard proxy configs for local testing (#30556) 2026-06-17 06:47:11 -07:00

litellm-proxy

A local, fast, and lightweight OpenAI-compatible server to call 100+ LLM APIs.

usage

$ uv tool install litellm
$ litellm --model ollama/codellama 

#INFO: Ollama running on http://0.0.0.0:8000

replace openai base

import openai # openai v1.0.0+
client = openai.OpenAI(api_key="anything",base_url="http://0.0.0.0:8000") # set proxy to base_url
# request sent to model set on litellm proxy, `litellm --model`
response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [
    {
        "role": "user",
        "content": "this is a test request, write a short poem"
    }
])

print(response)

See how to call Huggingface,Bedrock,TogetherAI,Anthropic, etc.


Folder Structure

Routes

  • proxy_server.py - all openai-compatible routes - /v1/chat/completion, /v1/embedding + model info routes - /v1/models, /v1/model/info, /v1/model_group_info routes.
  • health_endpoints/ - /health, /health/liveliness, /health/readiness
  • management_endpoints/key_management_endpoints.py - all /key/* routes
  • management_endpoints/team_endpoints.py - all /team/* routes
  • management_endpoints/internal_user_endpoints.py - all /user/* routes
  • management_endpoints/ui_sso.py - all /sso/* routes