scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-17 23:51:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 66

docs(AGENTS.md): add release branch CI lessons learned

Browse source 
Captures key patterns and pitfalls discovered during release branch
stabilization: enum types for mypy, -x flag masking, custom auth roles,
mock consistency, flaky test retries, transient provider errors, and
CircleCI infrastructure issues.

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
This commit is contained in:
Cursor Agent 2026-03-01 02:30:48 +00:00
parent a8cf646850
commit cc7671da96
1 changed files with 18 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

18
AGENTS.md
View file

@ -215,6 +215,24 @@ When opening issues or pull requests, follow these templates:
- Provider-specific docs in `docs/my-website/docs/providers/`
- Admin UI for testing proxy features
## RELEASE BRANCH CI LESSONS
When creating release branches and fixing CI failures, keep these hard-won lessons in mind:
1. **Always use proper enum types, not raw strings** — e.g., use `LitellmUserRoles.PROXY_ADMIN` instead of `"proxy_admin"` for `user_role` fields. MyPy CI checks enforce this and will fail on raw strings where enums are expected.
2. **CI `-x` flag hides downstream failures** — Many CI test suites run with `pytest -x` (stop on first failure). Fixing one test may unmask additional pre-existing failures in tests that were never reached before. Always check for this when a previously-failing suite starts passing its first test.
3. **Custom auth fixtures need admin roles for management endpoints** — The `custom_auth_basic.py` fixture (used in pass-through tests) must return `user_role=LitellmUserRoles.PROXY_ADMIN` for tests that call management endpoints like `/key/generate`. Without it, the route check in `non_proxy_admin_allowed_routes_check` rejects the request with 401.
4. **Mock patterns must match between sibling tests** — When a handler test (e.g., `test_streamable_http_mcp_handler_mock`) has a sibling (e.g., `test_sse_mcp_handler_mock`), they should use the same mock patterns. If one mocks `extract_mcp_auth_context` and `set_auth_context`, the other should too.
5. **Timing-sensitive tests need flaky retries in CI** — Tests that use `asyncio.sleep()` to wait for async callbacks (e.g., router usage tracking) need `@pytest.mark.flaky(retries=3, delay=1)` and generous sleep times (2s+) because CI runs on slower shared hardware.
6. **Transient provider errors are expected** — Gemini 503, websocket timeouts, and similar provider-side errors are transient and pass on rerun. Don't waste time fixing these — just rerun the workflow.
7. **CircleCI infrastructure can have capacity issues** — If ALL jobs fail immediately with "Task information unavailable" / canceled status, this is a CircleCI platform issue, not a code problem. Wait and retrigger.
## WHEN IN DOUBT
- Follow existing patterns in the codebase