* feat(router): support percentile-based TTFT routing
* fix(router): apply routing_strategy_args updates to the live selector
Runtime routing_strategy_args updates (config reload, update_settings)
only rebuilt the strategy selector when routing_strategy itself changed,
so a newly added ttft_percentile sat unused until the proxy restarted.
Also drops a comment that only restated the code it sat above.
Claude-Session: https://claude.ai/code/session_01PmqjhFYcUh6vA72d8W9gdB
* refactor(router): drop unreachable empty-samples guard in percentile latency
_percentile_latency is only called behind use_ttft, which already requires
a non-empty ttft sample list, so the early return was dead code and the one
line Codecov flagged as uncovered on this patch.
Claude-Session: https://claude.ai/code/session_01PmqjhFYcUh6vA72d8W9gdB
* test(router): cover the no-selector path of a routing_strategy_args update
simple-shuffle has no selector attribute to re-link, so the early return
guards a setattr with a None attribute name. Dropping the guard makes the
new test fail with "attribute name must be string, not 'NoneType'".
Claude-Session: https://claude.ai/code/session_01PmqjhFYcUh6vA72d8W9gdB
* fix(test): assert ValidationError on out-of-range ttft_percentile
pytest.raises(ValueError) tripped PT011 for being too broad. Pydantic
raises ValidationError for the gt/le constraint, so naming it satisfies
the rule and pins the assertion to the constraint under test.
Claude-Session: https://claude.ai/code/session_01PmqjhFYcUh6vA72d8W9gdB
* fix(router): drop Final from a per-deployment loop variable
basedpyright rejects "A Final variable cannot be assigned within a loop",
which pushed reportGeneralTypeIssues one over its budget. selected_latency
is rebound each iteration, so it matches its unannotated neighbours in the
same loop.
Claude-Session: https://claude.ai/code/session_01PmqjhFYcUh6vA72d8W9gdB
* test(router): exempt _apply_updated_routing_strategy_args from the name scan
The scan only reads test files with "router" in the filename, so it cannot
see the update_settings tests in router_strategy/test_lowest_latency.py.
Calling the private helper directly would test structure rather than
behaviour, so it joins the existing entries ignored for the same reason.
Claude-Session: https://claude.ai/code/session_01PmqjhFYcUh6vA72d8W9gdB
- fix(mypy): suppress [misc] type error in common_utils.py for cls.__init__ access
- fix(mypy): move type: ignore comment to correct line in test_eval.py (line 232 not 231)
- fix(mypy): suppress [misc] and pre-existing pyright errors in vertex_ai_non_gemini.py
- fix(check_licenses): strip inline comments before parsing requirements.txt lines so CVE comments don't break packaging.requirements.Requirement()
- fix(router_coverage): add _merge_tools_from_deployment and _invalidate_access_groups_cache to ignored list (private helpers tested indirectly)
- Add test_router_acancel_batch.py with mock test for router.acancel_batch()
- Add _acancel_batch to ignored list (internal helper tested via public API)
Fixes CI failure in check_code_and_doc_quality job
- Add test_get_valid_args in test_router_helper_utils.py to cover get_valid_args
- Use encoding='utf-8' in router_code_coverage.py for cross-platform file reads
* fix(cost_calculator.py): handle custom pricing at deployment level for router
* test: add unit tests
* fix(router.py): show custom pricing on UI
check correct model str
* fix: fix linting error
* docs(custom_pricing.md): clarify custom pricing for proxy
Fixes https://github.com/BerriAI/litellm/issues/8573#issuecomment-2790420740
* test: update code qa test
* fix: cleanup traceback
* fix: handle litellm param custom pricing
* test: update test
* fix(cost_calculator.py): add router model id to list of potential model names
* fix(cost_calculator.py): fix router model id check
* fix: router.py - maintain older model registry approach
* fix: fix ruff check
* fix(router.py): router get deployment info
add custom values to mapped dict
* test: update test
* fix(utils.py): update only if value is non-null
* test: add unit test
* refactor(router.py): move assistants api endpoints to using 1 pass-through factory function
Reduces code, increases testing coverage
* refactor(router.py): reduce _common_check_available_deployment function size
make code more maintainable - reduce possible errors
* test(router_code_coverage.py): include batch_utils + pattern matching in enforced 100% code coverage
Improves reliability
* fix(router.py): fix model id match model dump
