mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-15 23:31:29 +00:00
5ce28fb8e2
The Usage dashboard "Ask AI" feature posted the UI-selected model to /usage/ai/chat, and the backend called litellm.acompletion(model=...) directly. When the selected model is a configured proxy alias or model group, the bare SDK tried to parse the name as a raw provider/model string and the call failed, even though the same name works on /chat/completions. Every failure was collapsed into a generic "An internal error occurred", so the real cause was invisible. This replaces the whole usage_endpoints implementation with a v2 that routes the LLM call through the proxy's own llm_router, so model groups resolve and the call gets credentials, spend logging, budgets, rate limits, and guardrails like any other proxy request. Data access moves behind a ScopedUsageDataProvider whose authorization is baked in at construction: a non-admin caller gets a provider that can only ever read its own user_id, so a cross-tenant query is unrepresentable rather than something each tool has to remember to guard, and team/tag breakdowns are refused outright for non-admins. The tool loop now runs multiple rounds and streams the answer, terminal failures are modeled as a tagged union mapped to actionable messages, and the model comes from a usage_ai_model general setting rather than a hardcoded default. The SSE wire contract is unchanged, so the frontend panel is untouched apart from the model-selector placeholder that no longer advertises a hardcoded default. |
||
|---|---|---|
| .. | ||
| litellm-dashboard | ||
| Dockerfile | ||
| nginx.conf | ||