mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-07 08:26:10 +00:00
Merge pull request #32965 from BerriAI/litellm_pr_template_qa_runbook
docs(github): add QA runbook section to the PR template
This commit is contained in:
commit
b1de1812db
1 changed files with 24 additions and 0 deletions
24
.github/pull_request_template.md
vendored
24
.github/pull_request_template.md
vendored
|
|
@ -41,3 +41,27 @@ If you're seeing a delay in your PR being merged, ping the LiteLLM Team on [Slac
|
|||
✅ Test
|
||||
|
||||
## Changes
|
||||
|
||||
## QA runbook
|
||||
|
||||
<!-- Only needed when your PR edits tests/e2e; delete this section otherwise
|
||||
|
||||
For each e2e test you added or changed, list the manual steps a reviewer can follow to reproduce it by hand against a live proxy, mapping 1:1 to what the test asserts: one top-level bullet per test giving its pytest node id followed by what it proves in plain words, then a nested "- [ ]" checklist where each item is a concrete action (route, request body, expected response) and the final item is the sanity-check step shown in the examples. Note environment prerequisites (provider credentials, config flags) and any nuances a manual run will hit. See PRs #32914 and #32963 for full examples
|
||||
|
||||
Example checklists:
|
||||
|
||||
- tests/e2e/quota_management/ratelimit/test_rate_limit_e2e.py::TestKeyRateLimits::test_rpm_limit_blocks_over_limit - a key allowed 2 requests a minute serves exactly 2 and refuses the 3rd
|
||||
- [ ] Generate a limited key: curl -X POST http://localhost:4000/key/generate -H "Authorization: Bearer sk-1234" -d '{"rpm_limit": 2}'
|
||||
- [ ] Send three /v1/chat/completions requests with that key inside one minute
|
||||
- [ ] Expect the first two to return 200 and the third to return 429 naming the rpm limit
|
||||
- [ ] Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky
|
||||
|
||||
- tests/e2e/management/test_management_e2e.py::TestModelRoutes::test_model_create_appears_in_ui - a deployment created through the API shows up on the Admin UI models page
|
||||
- [ ] POST /model/new with the master key, a bedrock model, and aws_region_name (needs STORE_MODEL_IN_DB=True and AWS credentials)
|
||||
- [ ] Open http://localhost:4000/ui/?page=models and expect a deployment row showing the returned model id
|
||||
- [ ] Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky
|
||||
-->
|
||||
|
||||
### Final Attestation
|
||||
|
||||
- [ ] The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue