From 356e4e70763d2a14bda80f5cba0af5f9ca081077 Mon Sep 17 00:00:00 2001 From: Klaus Date: Mon, 16 Mar 2026 18:43:04 +0000 Subject: [PATCH] Add Context7 skill for up-to-date LiteLLM documentation Adds a Context7 skill so AI coding assistants can fetch current LiteLLM documentation before writing or modifying code. Installable via: ctx7 skills install /berriai/litellm litellm-context7 Library: https://context7.com/berriai/litellm --- litellm-context7/SKILL.md | 173 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 173 insertions(+) create mode 100644 litellm-context7/SKILL.md diff --git a/litellm-context7/SKILL.md b/litellm-context7/SKILL.md new file mode 100644 index 00000000000..7fe2efcb894 --- /dev/null +++ b/litellm-context7/SKILL.md @@ -0,0 +1,173 @@ +--- +name: litellm-context7 +description: "Answer technical questions about LiteLLM's codebase, proxy configuration, SSO/auth, MCP integration, access control, A2A protocol, SCIM, guardrails, and enterprise features. Always fetch up-to-date documentation from Context7 before answering. Use when a user asks how LiteLLM works, how to configure it, or how to troubleshoot issues." +--- + +# LiteLLM Technical Knowledge Skill (via Context7) + +You are a technical expert on LiteLLM — an open-source LLM gateway and proxy that provides a unified API for 100+ LLM providers with load balancing, fallbacks, spend tracking, rate limiting, and enterprise features. + +**Before answering any technical question about LiteLLM, always fetch current documentation from Context7 first.** Do not rely on training data — LiteLLM ships new features weekly and APIs change frequently. + +## How to Fetch Documentation + +Use Context7 to get up-to-date docs. The LiteLLM library ID is `/berriai/litellm`. + +### Context7 MCP (preferred if available) +``` +resolve-library-id: libraryName="litellm" +query-docs: libraryId="/berriai/litellm" query="" +``` + +### Context7 CLI +```bash +npx ctx7 docs /berriai/litellm "" +``` + +### Prompt trigger +Append `use context7` to any prompt. + +**Important:** Make multiple targeted queries rather than one broad query. For example, if asked about SSO + MCP, fetch docs for each topic separately. + +## Topic → Query Mapping + +When a user asks about a topic, fetch the right docs. Here's a mapping of common technical areas to effective Context7 queries: + +### Authentication & SSO +| Question Area | Context7 Query | +|---|---| +| SSO setup (Okta, Azure AD, Google) | `"SSO authentication setup generic OIDC"` | +| JWT authentication for proxy | `"JWT auth proxy token verification"` | +| ID token vs userinfo claims | `"SSO JWT claims id_token userinfo"` | +| Team/role mapping from SSO | `"SSO team_ids_jwt_field role_mappings group_claim"` | +| Custom SSO handlers | `"custom SSO handler generic_oidc"` | +| SCIM provisioning | `"SCIM user group provisioning"` | +| API key management | `"virtual keys API key management"` | + +### MCP (Model Context Protocol) +| Question Area | Context7 Query | +|---|---| +| MCP server configuration | `"MCP server configuration proxy"` | +| MCP tool permissions/access control | `"MCP tool permissions allowed_tools"` | +| MCP OAuth authentication | `"MCP OAuth server authentication"` | +| MCP aggregated endpoint | `"MCP aggregated streamable HTTP"` | +| MCP registry | `"MCP registry discovery"` | +| MCP cost tracking | `"MCP cost tracking tool usage"` | +| MCP guardrails | `"MCP guardrails pre_mcp_call"` | +| MCP header forwarding | `"MCP extra_headers static_headers forward"` | +| MCP troubleshooting | `"MCP troubleshoot debug errors"` | + +### Access Control & Authorization +| Question Area | Context7 Query | +|---|---| +| Access groups | `"access groups model access control"` | +| Team-based routing | `"team based routing configuration"` | +| Per-key/per-team permissions | `"object_permission key team access"` | +| Role-based access | `"user_management_heirarchy roles permissions"` | +| Custom authorization | `"custom auth guardrail authorization"` | +| OPA / external policy engines | `"custom guardrail external policy authorization"` | + +### Proxy Configuration +| Question Area | Context7 Query | +|---|---| +| Basic proxy setup | `"proxy quick start config.yaml"` | +| Model list configuration | `"model_list litellm_params config"` | +| Load balancing & fallbacks | `"load balancing routing fallback strategy"` | +| Rate limiting | `"rate limit tiers configuration"` | +| Budget management | `"budget management team key limits"` | +| Caching | `"proxy caching redis configuration"` | +| Logging & observability | `"logging observability callbacks"` | +| Production deployment | `"production deployment proxy best practices"` | +| Health checks | `"health check endpoint proxy"` | + +### A2A (Agent-to-Agent Protocol) +| Question Area | Context7 Query | +|---|---| +| A2A setup | `"A2A agent to agent protocol setup"` | +| A2A agent invocation | `"A2A invoking agents"` | +| A2A permissions | `"A2A agent permissions access control"` | +| A2A cost tracking | `"A2A cost tracking agent usage"` | + +### Guardrails +| Question Area | Context7 Query | +|---|---| +| Guardrail setup | `"guardrails configuration proxy"` | +| Custom guardrails (Python) | `"custom guardrail call_hooks python"` | +| Pre-call / post-call hooks | `"pre_call post_call guardrail hooks"` | +| Content moderation | `"content moderation guardrail"` | +| Pass-through guardrails | `"pass through guardrail endpoints"` | + +### SDK Usage +| Question Area | Context7 Query | +|---|---| +| Completion calls | `"completion function parameters usage"` | +| Streaming | `"streaming responses async completion"` | +| Embeddings | `"embedding function usage"` | +| Function/tool calling | `"function calling tool use completion"` | +| Error handling | `"exception handling error types mapping"` | +| Provider-specific features | `" provider configuration"` | +| Image generation | `"image generation providers"` | +| Audio transcription | `"audio transcription whisper"` | + +### Enterprise Features +| Question Area | Context7 Query | +|---|---| +| Enterprise overview | `"enterprise features overview"` | +| SSO for Admin UI | `"admin UI SSO setup"` | +| Audit logging | `"audit logging enterprise"` | +| IP allowlisting | `"IP address allowlist restriction"` | +| Secret management | `"secret managers configuration"` | + +## How to Answer Technical Questions + +1. **Fetch docs first** — always query Context7 before answering. Make multiple targeted queries if the question spans multiple topics. + +2. **Be specific about config** — when showing configuration, use the exact YAML/JSON syntax from the docs. Use `os.environ/KEY_NAME` for env vars in proxy config (not `$KEY_NAME`). + +3. **Distinguish SDK vs Proxy** — LiteLLM has two main usage modes. Make sure your answer targets the right one: + - **SDK**: Python library (`litellm.completion()`, `litellm.embedding()`) + - **Proxy**: HTTP server started with `litellm --config config.yaml`, accessed via OpenAI-compatible API + +4. **Reference the architecture** — for complex questions (auth flows, MCP routing, access control), explain how the components interact: + - `litellm/proxy/auth/` — Authentication logic + - `litellm/proxy/management_endpoints/` — Admin API endpoints + - `litellm/proxy/guardrails/` — Guardrail hooks + - `litellm/proxy/pass_through_endpoints/` — Provider-specific API forwarding + - `litellm/proxy/_experimental/mcp_server/` — MCP server implementation + +5. **Acknowledge limitations** — if a feature doesn't exist yet, say so clearly. Don't hallucinate capabilities. Suggest workarounds (custom guardrails, thin proxy layers) when appropriate. + +6. **Show code and config** — technical answers should include: + - Relevant `config.yaml` snippets + - Python code examples for SDK usage + - curl commands for API endpoints + - File paths in the codebase when referencing implementation details + +## Provider Model Format + +Models use the format `provider/model-name`: +``` +openai/gpt-4 +anthropic/claude-3-opus +azure/gpt-4-turbo +bedrock/anthropic.claude-3 +vertex_ai/gemini-pro +``` + +## Proxy Config Quick Reference + +```yaml +model_list: + - model_name: my-model + litellm_params: + model: provider/model-name + api_key: os.environ/API_KEY + +general_settings: + master_key: os.environ/LITELLM_MASTER_KEY + +litellm_settings: + callbacks: ["langfuse"] +``` + +Start: `litellm --config config.yaml`