Commit graph

4 commits

Author SHA1 Message Date
tavren27
89ca8d83f8
fix(ci): measure PR gates from the target branch's current tip (#49078)
* fix(ci): measure PR gates from the target branch's current tip

The lint gate base, the ui lint and ui unit test scopes, and the spend logs
guard compared the event's pull_request.base.sha with the head to find the
merge base. GitHub can leave base.sha weeks behind the base branch, and when
that old commit is an ancestor of a head that merged newer main, compare
returns the old commit itself, so the strict gates blamed main's own code on
the PR.

Compare base.ref with the head instead, passed through env, so GitHub
resolves the branch tip when the job runs. This is the same base make check
uses locally. The ui unit test push guard now keys on BASE_REF, which is
empty on push and workflow_dispatch.

The regression test runs every workflow step that reads merge_base_commit
against a fake gh backed by a temp repo with a stale base.sha, for main and a
litellm_/ base with a slash.

* fix(ci): check out the base branch's current guard code in the PR guards

* fix(ci): read the PR's current base branch so re-runs survive a deleted base

Every merge-base step now asks the pulls API for the PR's base branch instead of
reading the event's base.ref, which a re-run keeps after a stacked base is merged
and deleted. The guard checkouts go back to base.sha, which stays reachable.

* test(ci): mark the step script path Final

* fix(ci): keep the spend logs guard on main and run both guard scripts from the base branch tip

* fix(ci): measure from the event's base branch unless it was deleted, and test what each step hands on

A re-run after a retarget off main kept following the PR's current base, so lint, ui-unit and the cost map guard could pass without measuring against main. They now read the current base only when the event's base branch no longer exists. The tests run each merge base step whole and check the value its consumer gets

* fix(ci): follow a renamed base branch to its new name before the compare

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-29 18:28:16 -07:00
tavren27
e73f06ecdc
ci: run the dashboard API types and Terraform endpoint checks on every change that can alter the spec (#49022)
* ci: run the dashboard API types check on every change that can alter the OpenAPI spec

* chore: shorten the spec_pattern comment to a pointer at the CI filter

* chore: drop the spec_pattern comment

* docs: name the full ui-api-types trigger set in the make check header and dashboard AGENTS.md

* test: check make check and CI pick the same files for the API-type sync

* ci: run the Terraform provider endpoint audit on enterprise route changes

* test: keep the Terraform trigger test in the mapped workflow test file

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-29 07:19:06 -07:00
devin-ai-integration[bot]
a546a1720f
fix(ci): align misc unit tests with NativeCall bridge and widened e2e diff gates (#45180)
* fix(ci): pass NativeCall to transcription bridge fakes in rust bridge tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ci): assert the broadened e2e harness and basedpyright diff gates from #45172

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(rust-bridge): pin every NativeCall field in transcription bridge fakes

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-07 16:33:49 -07:00
yuneng-jiang
f6882246d4
test: move tests/test_litellm root and small trees into tests/unit (#43186)
* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: rename fork-flag to unit-flag now that it applies on every event

* test: move tests/test_litellm root and small trees into tests/unit

Pure renames, no content changes. Follow-up commits in this PR fix
references, merge the three files that already existed in tests/unit,
keep live-provider tests in tests/test_litellm and wire CI.

* test: carry tests/test_litellm conftest isolation into tests/unit

Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS,
proxy-URL and keychain env, and session-end client cleanup now reset for
unit tests too. The environment isolation owns its MonkeyPatch so a test's
own monkeypatch is undone before the model-cost teardown runs.

* test: merge, split and prune the moved root and small-tree tests

Merge batches/test_batch_utils.py and the chat_completions and messages
dispatch tests into the files that already existed in tests/unit. Keep
the live Gemini interactions tests, the async image-fetch format test and
the OpenAI embedding scorer test in tests/test_litellm since they need
real network or keys. Put test_router.py under tests/unit/test_router so
the existing package no longer shadows it. Delete eight tests the audit
found superseded by stronger ones kept in this move.

* ci: run the moved root and small-tree tests under their legacy flags

Add the misc and responses-caching-types flags to unit_selection.sh and
CircleCI, extend enterprise-routing and mcp-integration, and point the
legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest
and change classifier at the new paths.

* test: make the new tests/unit directories packages

tests/unit/test_package_layout.py requires every directory to carry an
__init__.py, and without one the moved and retained
test_litellm_responses_bridge.py modules collide on import.

* test: scope the unit socket block to tests/unit in shared sessions

The GHA shards collect the legacy test-path and the unit selection in one
pytest session. The unit conftest's loopback-only block leaked into legacy
modules that reach the network at import. The legacy conftest now lifts the
restriction at collect and setup time, and the unit conftest re-applies it
when collecting its own modules.

* test: give the shard-script tests their own GITHUB_OUTPUT

They only passed where the runner set it. The CircleCI unit job's env
allowlist drops it, so the script's redirect failed there.

* test: point the router and module-deletion checks at tests/unit

router_code_coverage and code_qa_check_tests only searched tests/test_litellm,
so the moved router tests no longer counted. The two silent-experiment tests
the audit deleted were the only direct callers of those methods; they are
replaced with tests that assert the forwarded shadow request and the
recursion guard.

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-25 11:30:43 -07:00
Renamed from tests/test_litellm/test_lint_workflow_diff_gates.py (Browse further)