- repository_dispatch (event_type: dbx-monitor) provides an external
trigger for price-drift alerting to kick off a check
- the job guard previously allowed only workflow_dispatch on the fork,
which would have skipped repository_dispatch runs; now only the
schedule event is pinned to upstream so cron never races, while both
manual trigger types run on either repo
The monitor used to run on the fork and open PRs against upstream,
which structurally requires a cross-repo PAT. Two changes remove it:
- PRs are created in whichever repo the workflow runs in
(GITHUB_REPOSITORY), so the built-in github.token covers both push
and pr create - same pattern as upstream's
auto_update_price_and_context_window workflow
- the job guard flips: upstream owns the merged schedule
(github.repository == BerriAI/litellm), the fork stays dispatch-only,
so the two never race on the same base
The script bootstrap from the fork branch stays until #38950 lands
upstream. DBX_MONITOR_TOKEN secret is no longer referenced.
Two fixes for the fork-run setup:
- the monitor audits upstream's registry (checkout pins
BerriAI/litellm@litellm_internal_staging), where the script does not
exist until #38950 lands - bootstrap it from the fork branch when
missing
- cron would fire on both repos once merged; guard the job to the fork
so only leecoder/litellm opens PRs and upstream never races
The monitor now executes on the leecoder/litellm fork:
- checkout pins the upstream repo and its litellm_internal_staging base,
so the monitor always audits upstream's registry, not the fork's
- schedule guard skips upstream's own checkout (only the fork runs it);
manual workflow_dispatch still works on upstream
- changed rates push to a dated branch on the fork and open the PR
against BerriAI/litellm with head leecoder:<branch>
- PR identity: leecoder / leecoder@aol.com
- auth uses the DBX_MONITOR_TOKEN secret (leecoder PAT with repo
scopes); the default GITHUB_TOKEN cannot create PRs across forks
- permissions trimmed to contents: write
The monitor hard-coded MONITORED = {deepseek-v4-flash, deepseek-v4-pro},
so new page models were never detected and column reordering could
silently mis-map rates.
- Parse every "Standard Pay Per Token" table on both pricing pages (open
FMS + proprietary FMS); map numeric columns by header text (Input /
Output / Cache read / Cache write), which differs per table
- Compare against every mapped registry entry and report UPDATED /
PROMO_SKIPPED / PROMO_ON_PAGE / REVIEW / RATES_AVAILABLE /
NOT_IN_REGISTRY / UNMAPPED_PAGE_MODEL / MISSING_FROM_PAGE
- Refresh only rate fields in place; metadata (context windows,
capabilities, deprecation dates) is preserved
- Cache fields: page value wins; when the page shows n/a, entries bill
cache at the input rate; custom conventions (gemini 0.1x reads) kept
- Dash/n-a cells are placeholders, not row qualifiers (gpt-oss / bge /
gemma rows were silently dropped before)
- Skip long-context tier rows and image/audio token sub-rows
- Workflow: PR body now embeds the full monitor report from
/tmp/dbx_monitor_pr_body.md
Verified live: on the current branch registry the monitor reports 10
cache-field UPDATEDs matching the values upstream already stores, 8
retired models flagged MISSING_FROM_PAGE, 2 PROMO_SKIPPED (gemini 2.5),
and NOT_IN_REGISTRY for every new model awaiting #39714.
- .github/scripts/assert_workflow_dir_hygiene requires only .yml files in
.github/workflows/; relocate monitor_databricks_pricing.py to scripts/
- Fix YAML ScannerError by turning the multi-line git commit -m block into
a single line (YAML block scalar containment issue)
- update the run step path to scripts/monitor_databricks_pricing.py
actions/checkout runs with persist-credentials: false, so the previous
'git push origin' would fail with auth error. Push over HTTPS with an
x-access-token URL using GITHUB_TOKEN. Addresses Greptile review:
'authenticated Git push configuration'.
- Add set -o pipefail so a scraper exception fails the job instead of being
misread as a price change
- Drop the || true suppression so a failed PR creation fails the run
- Pin actions/checkout and actions/setup-python to immutable commit SHAs
(repo convention, addresses Greptile/veria-ai review comments)
Adds a GitHub Action (daily 02:00 UTC + manual dispatch) that:
- Scrapes the official DBU rates for DeepSeek V4 Flash (0731) / V4 Pro (0813)
from https://www.databricks.com/product/pricing/foundation-model-serving
- Derives per-token USD at $0.07/DBU and updates both
model_prices_and_context_window.json and the packaged backup
- Opens a PR (base litellm_internal_staging) only when a rate actually changed
Runs on the leecoder fork (no repo guard) so the fork stays current and can
feed an upstream PR on demand.
* feat(skills): semantic search over the LiteLLM-hosted skill registry
Adds GET /v1/skills?query= (custom_llm_provider=litellm_proxy) and a
skill_search MCP virtual tool, ranking the caller's accessible skills by
semantic similarity, mirroring the A2A agent registry search (LIT-6309).
Also fixes a pre-existing bug where create_skill() dropped description and
instructions for the litellm_proxy provider, which left every LiteLLM-hosted
skill with no searchable text.
* fix(mcp): coerce skill_search top_k instead of raising 500 on malformed input
The MCP-REST skill_search dispatch validated raw tool arguments through a
pydantic model directly, so a non-numeric top_k raised a ValidationError
that the endpoint's catch-all turned into an HTTP 500. Mirrors the
agent_search branch's tolerant coerce_top_k handling instead.
* fix(skills): enforce key limits on search embeddings and bound the semantic index
Semantic search embeddings now run the same pre_call_hook the /embeddings
route runs, so key rate limits, budgets and guardrails apply before the
embedding model is called. The shared SemanticTextIndex caps cached vectors
and evicts the least recently searched entries, and each skill's embedded
text is capped so one skill cannot inflate the embedding batch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(skills): surface proxy 429s from search embeddings instead of a 503
ProxyRateLimitError is also an OpenAIError, so the search engine was folding
a key rate limit into skill_search_unavailable. Proxy HTTPExceptions now
propagate so the caller gets the same 429 the /embeddings route returns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(skills): import assert_never from typing_extensions for Python 3.10
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(skills): embed the request as the pre-call hooks returned it, not the original text
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(skills): keep the litellm_proxy provider check for GET /v1/skills?query= inside llms/
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(skills): move the GET /v1/skills?query= endpoint tests under tests/test_litellm/proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: limit Rust workflows to Rust directory changes
* ci: run Rust checks when their workflow changes
* ci: report Rust wheels only for successful Rust changes
* ci: keep Rust wheel reports in the workflow summary
* ci: group Rust lint and validation jobs
* ci: keep Rust job names distinct from required lint and test checks
* ci: drop the unused Python setup from the Rust lint job
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Its job id was `test`, the same check-run name test-mcp.yml publishes and
guard-internal-staging requires, so a commit touching the sweep carried two
check runs called `test`. Renaming it to `sweep-tests` leaves the required
context with exactly one job behind it.
A ruleset's required status check names a check run and GitHub matches it by
that name alone, so two jobs publishing the same name leave the gate unable to
say which job proved it. The new code-quality check reads every workflow,
expands matrix values and local reusable-workflow calls the way Actions does,
and fails when one name has more than one job behind it.
A harness-only change (proxy_client.py, conftest.py, pytest.ini, the gateway
config, .github/e2e-stack, or the workflow) selected nothing, so the stack was
never exercised by the change that touched it. select_tests.py keeps the
changed-file rule and adds the access_control suite whenever a harness file
changes. The run step now reports the pytest exit code before the evidence
check, prints pytest's summary line per pass so the rerun count is visible,
and assert_tests_ran.py names each failed or errored test as classname::name
The changed-tests workflow overrode the suite's `--reruns 1` with `--reruns 0`, so a
transport blip failed a pass that pytest.ini already scopes to network errors and
5xx responses. Pass 2 of run 33692484803 also went red 15s after a model write with
"no healthy deployments": the barrier only polled /v1/models through nginx, which
proves one gateway converged, and the next request rolled the other. The stack now
exports LITELLM_PROXY_REPLICA_URLS, the barrier polls every replica with the full
budget before settling, and up.sh refuses to boot without DD_API_KEY, since the
gateway config enables the datadog callback on every run
* fix(ci): grant pull_requests write for release wheel reporter
The reporter posts a PR comment via github.rest.issues.createComment.
GitHub requires both issues=write and pull_requests=write to comment on
a PR issue, as returned in x-accepted-github-permissions. The workflow
had pull-requests: read, so the POST failed with 403 'Resource not
accessible by integration'. Bumping to pull-requests: write fixes the
create path; the read-only pulls.get call still works. Same-repo scope
is preserved by the existing head_repository.full_name check.
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
* fix(ci): scope release wheel reporter permissions to pull requests
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
Co-authored-by: Yujong Lee <yujong@berri.ai>
Replace test-model-map.yml with a pull_request_target guard that validates the
cost map, its backup, and its generated schema on every PR, and additionally
enforces the sync bot contract on litellm_cost_map_sync_* branches: only the
three cost map files may change, no model or field is removed, and the special
root keys stay untouched.
Clippy never links, so python-config's pyo3/auto-initialize needs no
libpython and the gateway clippy step can cover every feature at once.
The test step stays on --features server because cargo test does link
and this job installs no Python.
The check list existed in three places that had already drifted apart;
CLAUDE.md is now the only copy and the other two point at it.
The build-ui check compiled the dashboard from a full checkout, so any
import reaching above ui/litellm-dashboard/ resolved there and only broke
inside the images, where the stage copies the dashboard tree alone.
Building the stage itself puts the check on the same file boundary the
shipped images use.
litellm-ai-gateway's server feature is off by default and nothing in the workspace turns it on, so the workspace clippy and test steps never compiled src/auth, src/routes, src/state, src/realtime or the gateway binary. 43 tests ran instead of 57.
Adds the two steps CLAUDE.md already documents as the local gate, and fixes the three collapsible_if violations that had accumulated behind the flag.
Forwarding limit to OpenAI made the ownership filter cut the page down after
the fact, so a key that owned an older container got an empty first page and
its cursor never moved. Non-admin lists now walk upstream pages of 100 until
they have enough owned containers (or five pages), trim to the requested
limit, and report first_id, last_id and has_more off what the caller keeps.
Also assigns tests/test_litellm/proxy/container_endpoints to a CI shard.
* refactor(python-bridge): split non-streaming bridge modules
* refactor(python-bridge): bring shared function tracing into route layer
* feat(dev): list Python route functions and call sites
* feat(dev): list Rust route functions and call sites
* docs(dev): record OCR parity gaps across Python and Rust
* feat(dev): list executed SDK calls with runtime tracing
* feat(dev): report Python vs Rust SDK pipeline steps in one CLI
* feat(dev): side-by-side pipeline step report in compare CLI
* fix(dev): drop invalid Final annotations in compare cell loop
* feat(dev): blue python-only and yellow rust-only steps in compare CLI
* feat(dev): vertical layout with section spacing in compare CLI
* fix(dev): validate SDK trace stages across sync and async routes
* refactor(rust): align SDK route call structure with Python
* refactor(python-bridge): share sync and async route call wrappers
* refactor(dev): split compare CLI into fixtures, runtime, and report modules
* fix(ci): run SDK trace tests and satisfy test lint
Adds a required-check candidate that selects the tests/e2e test files a PR added or
modified, boots a stage-mirror stack on the runner (migrations, backend, two gateway
processes behind nginx, Postgres, Jaeger, TLS cluster Valkey), and runs those files
three times with retries off. The run job sits behind the e2e-changed GitHub
environment, so a reviewer approves each run before the OIDC token that reads the
provider keys from AWS Secrets Manager exists. Supersedes #34981
