mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
ae8084de74
* ci(codspeed): pin benchmark runner to ubuntu-24.04 ubuntu-latest resolves to different runner images between the BASE (main/staging) and HEAD (PR) runs, so CodSpeed reports 'Different runtime environments detected' and emits false-positive regressions (e.g. a -25.2% swing on test_completion_multi_turn in #31684, an MCP auth fix with no LLM code changes). Pinning the runner to a fixed image keeps BASE and HEAD on the same hardware so 1 ms swings on a ~3 ms benchmark stop blocking unrelated PRs. Fixes #31738 * ci(codspeed): stop running benchmarks on litellm_internal_staging The CodSpeed check flip-flops on internal staging and on PRs targeting it (e.g. "+11.75% improvement" on one run, "-25.36% regression" on the next) because the comparison flags "different runtime environments" and the benchmarks are only 3-4 ms, so sub-millisecond runner noise swings the result by 25-30%. Pinning the runner to ubuntu-24.04 in this PR helps the head side, but the internal_staging base is still recorded on the old unpinned runner, so comparisons keep flapping until the pin merges and the base is re-baselined. Until that settles, the red X's on internal staging make the OSS project look unhealthy and confuse contributors, so drop the litellm_internal_staging push and pull_request triggers and keep CodSpeed running on main only. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| codeql | ||
| ISSUE_TEMPLATE | ||
| observatory | ||
| screenshots | ||
| scripts | ||
| workflows | ||
| dependabot.yaml | ||
| deploy-on-aws.png | ||
| deploy-on-gcp.png | ||
| deploy-to-aws.png | ||
| FUNDING.yml | ||
| pull_request_template.md | ||
| template.yaml | ||
