litellm/.github/workflows/cost-map-pins-recheck.yml
devin-ai-integration[bot] ec7162aea2
fix(cost_map): mark fireworks_ai/minimax-m3 as vision and pin capabilities verified against live calls (#43390)
* fix(cost_map): mark fireworks_ai/minimax-m3 as vision and pin capabilities verified against live calls

Fireworks' serverless minimax-m3 answers image inputs, but its model
listings say supports_image_input false, so the last two registry syncs
flipped supports_vision back to false and the Fireworks pre-flight turned
every image request into a 400 without ever calling the provider.

Flip both minimax-m3 rows back to supports_vision true and record the
live-call evidence in ci_cd/cost_map_pins.json. The cost map guard now
fails any PR that lowers a pinned capability, quoting the evidence, and
the sync bot may not touch the pins file.

* fix(cost_map): fail the guard when the pins file is missing

* fix(cost_map_guard): skip the pin checks on a branch cut before the pins file existed

* fix(ci): check cost map pins against the merge result and keep price sync PRs off the pins file

* fix(ci): run the cost map guard against the live base branch tip and re-run it when main's pins change

* fix(ci): pass the live base tip to the cost map guard through env and re-run every guard run on a pin change

main now runs the guard on pull_request with the base branch's copy of the script, so the workflow passes the
live tip in COST_MAP_GUARD_TARGET, which the older script ignores. The recheck reads past GraphQL's 100-file
cap, re-runs every guard run for a shared head, and fails when a re-run cannot start. The guard tells a merge
conflict apart from a failed merge-tree call.

* fix(ci): run the cost map pins recheck on litellm_ base branches too

* fix(ci): bar bot-opened PRs from the pins, recheck pin PRs on cost map changes, and skip the merge for untouched PRs

* fix(ci): read the pushed files from the compare API in the cost map pins recheck

The push event GitHub hands to Actions leaves each commit's added, modified and
removed lists null, so the recheck never saw a pins change and re-ran only the PRs
that touch the pins. Compare the push's before and after commits instead, and
re-check every cost map PR when the branch is new, the push was forced, the
compare fails, or it lists 300 files, the most it returns.

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-29 00:23:11 -07:00

91 lines
4.2 KiB
YAML

name: Cost map pins recheck
on:
push:
branches:
- main
- "litellm_**"
paths:
- model_prices_and_context_window.json
- litellm/model_prices_and_context_window_backup.json
- ci_cd/cost_map_pins.json
permissions: {}
jobs:
rerun-cost-map-guard:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
actions: write
contents: read
pull-requests: read
steps:
- name: Re-run the cost map guard on open pull requests the push could break
env:
GH_TOKEN: ${{ github.token }}
BASE_REF: ${{ github.ref_name }}
BEFORE_SHA: ${{ github.event.before }}
AFTER_SHA: ${{ github.event.after }}
FORCED: ${{ github.event.forced }}
COST_MAP_PATHS: '["model_prices_and_context_window.json", "litellm/model_prices_and_context_window_backup.json", "ci_cd/cost_map_pins.json"]'
PINS_PATHS: '["ci_cd/cost_map_pins.json"]'
run: |
if [[ "$BEFORE_SHA" =~ ^0+$ || "$FORCED" == true ]] \
|| ! gh api "repos/${GITHUB_REPOSITORY}/compare/${BEFORE_SHA}...${AFTER_SHA}?per_page=1" \
--jq '(env.PINS_PATHS | fromjson) as $pins | (.files | length) < 300 and all(.files[]; [.filename, .previous_filename] | any(IN($pins[])) | not)' \
| grep -qx true; then
export RECHECK_PATHS="$COST_MAP_PATHS"
else
export RECHECK_PATHS="$PINS_PATHS"
fi
rerun_cutoff="$(date -u -d '30 days ago' +%Y-%m-%dT%H:%M:%SZ)"
candidates="$(gh api graphql --paginate -F owner="${GITHUB_REPOSITORY_OWNER}" -F name="${GITHUB_REPOSITORY#*/}" -F base="$BASE_REF" \
-f query='query($owner: String!, $name: String!, $base: String!, $endCursor: String) {
repository(owner: $owner, name: $name) {
pullRequests(states: OPEN, baseRefName: $base, first: 25, after: $endCursor) {
nodes { number headRefOid changedFiles files(first: 100) { nodes { path } } }
pageInfo { hasNextPage endCursor }
}
}
}' \
--jq '(env.RECHECK_PATHS | fromjson) as $paths | .data.repository.pullRequests.nodes[]
| {number, headRefOid, listed: any(.files.nodes[].path; IN($paths[])), unlisted: (.changedFiles > 100)}
| select(.listed or .unlisted) | "\(.number) \(.headRefOid) \(.listed)"')"
failed=0
declare -A rerun_heads
while read -r number head_sha listed; do
[ -n "$number" ] || continue
if [ "$listed" = false ]; then
if ! changed="$(gh api --paginate "repos/${GITHUB_REPOSITORY}/pulls/${number}/files?per_page=100" \
--jq '(env.RECHECK_PATHS | fromjson) as $paths | .[].filename | select(IN($paths[]))')"; then
echo "::error::could not list the files of #$number"
failed=1
continue
fi
[ -n "$changed" ] || continue
fi
[ -z "${rerun_heads[$head_sha]:-}" ] || continue
rerun_heads[$head_sha]=1
if ! runs="$(gh run list --repo "$GITHUB_REPOSITORY" --workflow cost-map-guard.yml --commit "$head_sha" --limit 100 --json databaseId,createdAt --jq '.[] | "\(.databaseId) \(.createdAt)"')"; then
echo "::error::could not list the cost map guard runs of #$number"
failed=1
continue
fi
if [ -z "$runs" ]; then
echo "::warning::no cost map guard run found for #$number"
continue
fi
while read -r run_id created_at; do
if [[ "$created_at" < "$rerun_cutoff" ]]; then
echo "::warning::cost map guard run $run_id for #$number is over 30 days old, so GitHub cannot re-run it; #$number gets the pin checks on its next push"
continue
fi
gh run watch "$run_id" --repo "$GITHUB_REPOSITORY" > /dev/null || true
if ! gh run rerun "$run_id" --repo "$GITHUB_REPOSITORY"; then
echo "::error::could not re-run cost map guard run $run_id for #$number"
failed=1
fi
done <<< "$runs"
done <<< "$candidates"
exit "$failed"