mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-05 08:07:05 +00:00
8536e3b80e
* fix(proxy): source /v1/models token limits from cost map instead of Router.get_model_group_info Resolves the per-model get_model_group_info fan-out on GET /v1/models (and /models) that pegged the event loop on wildcard listings (#33636). create_model_info_response now reads max_input_tokens/max_output_tokens from litellm.get_model_info (the static cost map) rather than the router, which aggregated and deepcopied every deployment in a group per listed model. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): inject model-info lookup into create_model_info_response for deterministic coverage Inject the cost-map lookup (defaulting to litellm.get_model_info) so the except and max_output_tokens branches are exercised deterministically and the token-limit tests no longer hardcode mutable cost-map values. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(proxy): surface custom deployment token limits on /v1/models via cheap index lookup Add Router.get_configured_token_limits, an O(1) model-name index lookup that reads a concrete deployment's configured max_input_tokens/max_output_tokens without triggering pattern matching or deep copies. create_model_info_response layers this over the cost map so custom deployments absent from the cost map still surface their limits, and admin-configured limits override cost-map defaults, while wildcard-expanded names stay on the fast path. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: ryan <ryan@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| helpers | ||
| prisma_and_spend | ||
| proxy_logging | ||
| __init__.py | ||