* fix(proxy): source /v1/models token limits from cost map instead of Router.get_model_group_info
Resolves the per-model get_model_group_info fan-out on GET /v1/models
(and /models) that pegged the event loop on wildcard listings (#33636).
create_model_info_response now reads max_input_tokens/max_output_tokens
from litellm.get_model_info (the static cost map) rather than the router,
which aggregated and deepcopied every deployment in a group per listed
model.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): inject model-info lookup into create_model_info_response for deterministic coverage
Inject the cost-map lookup (defaulting to litellm.get_model_info) so the
except and max_output_tokens branches are exercised deterministically and
the token-limit tests no longer hardcode mutable cost-map values.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): surface custom deployment token limits on /v1/models via cheap index lookup
Add Router.get_configured_token_limits, an O(1) model-name index lookup that
reads a concrete deployment's configured max_input_tokens/max_output_tokens
without triggering pattern matching or deep copies. create_model_info_response
layers this over the cost map so custom deployments absent from the cost map
still surface their limits, and admin-configured limits override cost-map
defaults, while wildcard-expanded names stay on the fast path.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy/utils): pin bottom-of-file helper behavior
Pin current behavior of the bottom-of-file pure-function helpers in
litellm/proxy/utils.py (projection, team config, time helpers,
guardrail merge, error helpers, URL/path helpers, premium gate, model
access, and misc DB/API-key helpers).
Adds tests/test_litellm/proxy/utils/helpers/ with one happy + one error
test per pinned symbol; folds the prior single-test
tests/test_litellm/proxy/test_utils.py into test_url_helpers.py and
deletes the old file. _pin_check.py and _coverage_check.py serve as
local stopping gates.
Adds tests/test_litellm/proxy/utils to the existing test-path block in
.github/workflows/test-unit-proxy-endpoints.yml.
Plan: https://www.notion.so/37343b8acdab81f68f39f66915f62bcf
Pin list: https://www.notion.so/37343b8acdab8150acdbf40e5756869f
* test(proxy/utils): apply greptile fixes to behavior-pinning gates
Address findings from the sibling PR1/PR2 greptile reviews that also
apply to this PR:
- Commit pin_list.txt alongside the gate script (was previously a
gitignored .pin_list.txt fetched from Notion). The gate is now
reproducible without out-of-band setup.
- Resolve the coverage region by locating the first pinned symbol's
def line in litellm/proxy/utils.py at runtime, instead of hardcoded
line numbers that drift when lines above shift.
- Word-boundary the pin reference check so pins like update_spend do
not falsely match update_spend_logs_job.
- Drop the dead _harness_smoke_test.py exclusion; the test_*.py glob
already filters underscore-prefixed files.
* test(proxy/utils): drop local-only stopping-signal scripts
Remove _pin_check.py, _coverage_check.py, and pin_list.txt. These were
dev-time tooling for knowing when test authoring was done; they are
not wired into CI and the test files themselves are the merge artifact.
---------
Co-authored-by: Claude <noreply@anthropic.com>
