GitHub expressions have no arithmetic operators, so
`${{ inputs.timeout-minutes + inputs.setup-timeout-minutes }}` was not a value
but a startup failure. The proxy-db workflow died before creating any job on
both prior commits, which posts no check run at all: the entire suite stopped
running while the PR's checks stayed green.
Pass the job backstop in as `job-timeout-minutes` instead of computing it, and
size it as the test budget plus the 30 minutes of setup ceilings plus 5 minutes
of runner overhead the job clock charges but no step owns.
check_workflow_startup_safety.py makes this class of mistake visible before
merge, since CI cannot report it: it rejects arithmetic inside an expression
and checks every caller of the reusable workflow keeps a job budget large
enough that the deadline cannot preempt pytest inside its own budget.
PR #36166 added tests/test_litellm/proxy/credential_endpoints/test_endpoints.py
but no CI job invokes it, so the CI Coverage guard failed on
litellm_internal_staging. Add the directory to the proxy-endpoints
job's test-path list so pytest actually runs the new tests and the
coverage assertion is satisfied.
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
The absence of a CI signal is indistinguishable from a passing one, and
that shape has now produced several independent holes: whole test
directories no job runs, and shipped images no job builds. Nothing was
watching for either, so each was found by accident.
assert_ci_coverage.py enumerates every test_*.py under tests/ and every
Dockerfile in the repo, then credits only what the workflows and the
CircleCI config actually invoke. Paths filters and lint steps that merely
name a directory do not count as coverage, because crediting a mention is
the same mistake one level up. Anything neither invoked nor listed in
.github/ci-coverage-allowlist.yml with a written reason fails the job.
The guard found 275 uncovered test files and 6 unbuilt Dockerfiles. Fixed
here: the root Dockerfile, the primary published image, now gets an
image-scan leg that builds it and runs the offline migration check against
it, and 8 tests/test_litellm subdirectories join the shards that already
enumerate their siblings. Everything else is allowlisted per file, so a
new file cannot inherit an exemption, and the remaining decisions are
tracked rather than invisible.
The job reports but is not in branch protection, so it does not block
merges; promoting it is a separate change once the allowlist has survived
contact with a few pull requests.
The unit workflows trigger on pull_request only, so a commit that
actually lands on a gated branch ends up with no unit-test check runs at
all. The commit status API reports success for those commits, which a
release gate reads as "nothing failed" rather than "never tested". PR
checks also only ever ran against the merge preview, not the commit that
landed, so two branches that are each green can still land broken
together
Add a push trigger on the two gated branches to the twelve unit
workflows and to the code-quality workflow, so every landed commit gets
check runs addressable by its SHA
test-linting.yml is deliberately left on pull_request only; six of its
steps gate on a diff against github.event.pull_request.base.sha, which
is empty outside a pull request, and a "what did this branch add" check
has no meaning on a merge commit
Also key the concurrency group on github.sha and restrict
cancel-in-progress to pull_request. The previous group was stable across
pushes to a branch, so consecutive merges would cancel the in-flight run
for the earlier commit and leave that SHA without a result, which is the
same blind spot this change is meant to close
The SSO and Email Server settings pages read only stored config, so a gateway
configured entirely through environment variables rendered every field blank
even though both features were live. Rather than add per-endpoint env fallback,
resolve each setting through one typed config object.
A FieldDescriptor names, for one setting, where it lives in the stored row
(db_key), which process env var carries it (env_var), whether it is a secret,
and its effective default. A pure resolve_fields reconciles a descriptor table
against the stored row and the process environment with a fixed precedence and
reports per-field provenance (db, env, default, or unset). The SSO descriptor
table single-sources the field-to-env mapping that the read and write paths
previously duplicated, so they can no longer drift.
get_sso_settings and the /get/config/callbacks alerting block read through the
resolver instead of their own inline fallbacks. get_sso_settings no longer
decrypts stored values into os.environ; decryption happens once inside the
resolver via the pure helper, so a GET stops mutating the process environment.
The SSO response carries provenance so the UI can distinguish an env-sourced
value from a stored one, and secrets are masked at the endpoint (the resolver
returns them unmasked so the login path could consume them). os.environ remains
the runtime carrier; the SSO login and mail-send paths are unchanged.
The settings pages also submit only fields an admin actually edited, so a
rendered mask or env-sourced value is never written back over a working
secret, and generic_scope is a real SSO form field. Omitting a field from
/update/sso_settings clears it, which provider switching relies on; the deeper
write-path concern that behaviour points at is tracked in LIT-4498.
* ci: skip unit test workflows when only docs or ui files change
Mirror the CircleCI backend path filter (.circleci/scripts/classify_changes.sh)
in the GitHub Actions unit test workflows by adding paths-ignore for ui/**,
docs/**, *.md and *.mdx to every test-unit-*.yml pull_request trigger
* ci: drop docs/** from unit test paths-ignore since the folder no longer exists
* test(batches): add 1:1 test file scaffold for batches component paths
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add harness test for create batch endpoint
* Add retrieve endpoint harness tests
* Add list endpoint harness tests
* Add cancel endpoint harness tests
* Add cancel endpoint harness tests
* Add test for litellm/batches/main.py
* Add test for litellm/tests/test_litellm/batches/test_batch_utils.py
* Add handler and transformation tests for all providers
* Fix: run batches tests in cicd
* fix(tests): remove azure/__init__.py that shadowed azure namespace package
Adding __init__.py to tests/test_litellm/llms/azure/ caused pytest to
insert tests/test_litellm/llms/ into sys.path[0], making our empty
azure/ dir shadow the real azure-identity namespace package. Any test
that patched azure.identity.* would then fail with AttributeError.
* style(tests): apply ruff format to test_batch_utils.py
Base migrated the formatter from black to ruff format (#31317); reformat the
batches scaffold test file to match.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Keep only video test files and CI workflow entries; drop unrelated
production code and non-video test changes from this branch.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ci): point OSS contributor workflows to litellm_oss_staging
Workflow triggers and guard error messages incorrectly referenced litellm_oss_branch; update them to the branch we actually use for external contributions.
* fix(ci): include test-rust.yml in litellm_oss_staging rename
Missed test-rust.yml when updating OSS contributor target branch references.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Switch the zizmor check to fail on any finding at medium severity or
above (advanced-security off, min-severity medium, annotations on) so it
can be promoted to a required check, and pin the engine to zizmor 1.24.1
through zizmor-action v0.5.6 for deterministic runs.
Clear the findings that were outstanding so the check passes: correct
mismatched action pin version comments, scope the proxy endpoint
workflow's id-token and pull-requests permissions to the jobs that use
them, and mark the server-root-path docker build as non-publishing while
dropping its shared gha build cache.
* test(proxy/utils): pin bottom-of-file helper behavior
Pin current behavior of the bottom-of-file pure-function helpers in
litellm/proxy/utils.py (projection, team config, time helpers,
guardrail merge, error helpers, URL/path helpers, premium gate, model
access, and misc DB/API-key helpers).
Adds tests/test_litellm/proxy/utils/helpers/ with one happy + one error
test per pinned symbol; folds the prior single-test
tests/test_litellm/proxy/test_utils.py into test_url_helpers.py and
deletes the old file. _pin_check.py and _coverage_check.py serve as
local stopping gates.
Adds tests/test_litellm/proxy/utils to the existing test-path block in
.github/workflows/test-unit-proxy-endpoints.yml.
Plan: https://www.notion.so/37343b8acdab81f68f39f66915f62bcf
Pin list: https://www.notion.so/37343b8acdab8150acdbf40e5756869f
* test(proxy/utils): apply greptile fixes to behavior-pinning gates
Address findings from the sibling PR1/PR2 greptile reviews that also
apply to this PR:
- Commit pin_list.txt alongside the gate script (was previously a
gitignored .pin_list.txt fetched from Notion). The gate is now
reproducible without out-of-band setup.
- Resolve the coverage region by locating the first pinned symbol's
def line in litellm/proxy/utils.py at runtime, instead of hardcoded
line numbers that drift when lines above shift.
- Word-boundary the pin reference check so pins like update_spend do
not falsely match update_spend_logs_job.
- Drop the dead _harness_smoke_test.py exclusion; the test_*.py glob
already filters underscore-prefixed files.
* test(proxy/utils): drop local-only stopping-signal scripts
Remove _pin_check.py, _coverage_check.py, and pin_list.txt. These were
dev-time tooling for knowing when test authoring was done; they are
not wired into CI and the test files themselves are the merge artifact.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat(a2a): well-known agent-card discovery + LangGraph Platform mode
Adds a registration-time discovery flow so admins can paste an upstream
agent URL, see its skills/capabilities, pick what to expose, and have the
proxy front it with a LiteLLM-shaped agent card.
Backend (new litellm/proxy/a2a/ module):
- fetch_well_known_card walks /.well-known/agent-card.json,
/.well-known/agent.json, /agent.json by default. langgraph_platform
mode hits the canonical path with ?assistant_id=<id> (LangGraph
serves one shared endpoint per deployment).
- merge_agent_card overlays LiteLLM overrides on the upstream card:
drops upstream url, forces protocolVersion=1.0, replaces
securitySchemes with LiteLLMKey bearer, emits supportedInterfaces
pointing at the proxy, filters capabilities to a small allowlist,
strips non-v1.0 fields.
- POST /v1/a2a/discover returns the raw upstream card (admin-only) so
the UI can render skills/capabilities for selection.
- create/update/patch agent endpoints pre-generate the agent_id and
run merge_agent_card before storing, so DB.agent_card_params already
embeds the proxy-fronted URL.
UI (ui/litellm-dashboard):
- New AgentCardDiscovery component with a parent-driven plan:
discovery_mode + params + display URL. For LangGraph the parent
composes (api_base, assistant_id); for pure A2A it uses the url
field. Component hides the manual URL input when the parent drives.
- add_agent_form wires discovery for every non-custom agent type and
overlays the user's selections onto agent_card_params at submit,
fixing the bug where dynamic agent forms ignored discovery picks.
Completion-bridge fixes (paired):
- Add kind: "message" to A2A response messages and unwrap result
so it's a Message directly per spec (matches a2a SDK
SendMessageResponse validation).
- Forward A2A metadata to LangGraph runs via extra_body.metadata.
* fix(a2a): preserve agent url, fix streaming chunk envelope, and protect forwarded metadata
- Streaming chunk: move final out of the message object into the
result envelope per the A2A spec.
- Agent card merge: keep upstream url on the stored card so the
runtime invocation path can locate the upstream backend; the public
well-known endpoint already rewrites this field to the proxy URL
before exposing it to clients.
- Completion bridge: apply A2A forward metadata after merging
litellm_params so an agent-configured extra_body cannot
overwrite the forwarded metadata.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(a2a): fix legacy streaming chunk, agent card test, and metadata merge
- providers/litellm_completion: move 'final' out of the message object
into the result envelope per the A2A spec (matches the bridge fix).
- agent endpoints test: the runtime invocation path now preserves the
top-level 'url' on the stored card, so update the assertion to match.
- completion bridge metadata: when forwarding A2A metadata via
extra_body.metadata, merge into any existing extra_body.metadata
instead of replacing it, so an agent-configured metadata block is
preserved (forward metadata still wins on key conflicts).
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(a2a): remove dead duplicate transformation dir; drop SSRF-prone headers field from /v1/a2a/discover
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(a2a): revert accidental html→index.html rename from afc8b10f
The commit afc8b10f bundled real A2A fixes alongside an unintended
re-introduction of the */index.html layout that 8513d7fc had already
reverted. Restore all 35 static-export pages back to the flat *.html
structure that matches the upstream main branch.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(a2a): address PR review comments
UI:
- Auto-trigger discovery when connection details are filled; remove
the "Use these selections" button (selection syncs live to parent,
user just clicks Next).
- Edit Settings: auto-discover upstream card on open; cross-check with
DB-stored card so only already-saved skills/capabilities are pre-ticked.
- Extract shared buildDiscoveryRequest + selectionsFromSavedAgentCard
helpers into agent_discovery_utils.ts so both add and edit flows share
the same logic.
Backend:
- agent_card.py: rename the proxy security requirements field from the
non-standard ``securityRequirements`` to the spec-correct ``security``
key (matches AgentCard TypedDict and A2A/OpenAPI convention).
- agent_card.py: remove ``securityRequirements`` from _ALLOWED_TOP_LEVEL_KEYS.
- endpoints.py: _build_merged_agent_card now forwards agent_name and
description from the request so the stored card reflects the admin-
supplied name, not just whatever the upstream card advertised.
- utils.py: remove overly-broad ``or "parts" in result`` fallback; use
``kind == "message"`` check only to avoid false matches on future
result types that happen to include a ``parts`` field.
- test_agent_card.py: update assertions to expect ``security`` key.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: restore Next.js metadata directories to match upstream main
The previous revert removed __next.* metadata subdirectories from git
tracking entirely, but these directories exist on origin/main alongside
the flat .html files. Restore them via checkout from origin/main so the
PR diff only reflects actual code changes.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(a2a): drop dead headers option from discoverAgentCardCall
The backend /v1/a2a/discover endpoint no longer accepts a headers field
(removed in 78591b2 for SSRF safety), so any headers passed through
DiscoverAgentCardOptions were silently discarded by the API request
body. Remove the field and the conditional that copies it onto the
request body.
* fix(a2a): skip merge for non-A2A agents and align pydantic-ai result shape
The agent create/update/patch handlers ran the LiteLLM-fronting merge
unconditionally, so registrations that did not provide
agent_card_params still ended up with a synthesised card carrying
supportedInterfaces, securitySchemes, and default skills. Gate the
merge on a non-empty agent_card_params so plain chat/LLM agents stay
non-A2A in the registry.
Also move kind: 'message' inside the a2a_message dict in the Pydantic
AI non-streaming response so its construction matches the completion
bridge rather than spreading kind on top of a separate dict.
* Fix three bugs in A2A discovery flow
1. UI: Stabilize discoveryRequest deps to avoid redundant /v1/a2a/discover
API calls. The parent rebuilds the discoveryRequest object on every form
keystroke, so depend on primitive proxies (discovery_mode + serialized
params) rather than the object identity. Read the actual object via a
ref inside handleDiscover.
2. Backend: Route the well-known card fetch through async_safe_get so the
admin /v1/a2a/discover endpoint can't be used to probe private/loopback
addresses or cloud metadata endpoints. SSRFError is a separate handled
case so it surfaces a clear AgentCardDiscoveryError.
3. Streaming: Make openai_chunk_to_a2a_chunk emit the same flat result
shape as the non-streaming response (kind/role/parts/messageId at the
result level), with envelope-level 'final' added. Matches the existing
create_artifact_update_event pattern and lets consumers read a uniform
result shape across streaming and non-streaming.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(a2a/ui): include savedAgentCard in handleDiscover deps
The previous deps list omitted savedAgentCard, so handleDiscover (and
the resetSelections it calls) kept the closure's saved-card value even
after the parent refetched the agent. Clicking 'Re-discover' would
then pre-select skills against stale data. Adding savedAgentCard to
the deps array forces the callback to refresh whenever the saved card
changes.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(a2a): align pydantic-ai test + docstring with direct-Message result shape
The non-streaming A2A response was changed so that `result` is the Message
itself (kind="message"), per spec / SendMessageResponse. Update the
PydanticAITransformation._transform_to_a2a_response test and docstring that
still described the old `result.message` envelope so internal consumers
match the producer.
* fix(a2a): strip additionalInterfaces and let configured metadata win over A2A request
- merge_agent_card no longer carries upstream additionalInterfaces through;
storing those alternate URLs would let authenticated agent callers reach
the backend directly and bypass proxy auth/budget/logging.
- apply_forward_metadata_to_completion_params now layers client-supplied A2A
metadata UNDER any agent-owner-configured extra_body.metadata, so server-set
run metadata stays authoritative on key conflicts.
* fix(agents): merge agent card even when agent_card_params is an empty dict
Treat an explicitly provided empty agent_card_params ({}) as 'card
provided but empty' instead of 'no card', so the LiteLLM-fronting merge
still injects securitySchemes, supportedInterfaces, and protocolVersion.
Without this, the well-known endpoint could serve a bare card with only
a rewritten url, advertising no authentication to A2A clients.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* refactor(a2a): drop dead openai_chunk_to_a2a_chunk helper
The deprecated single-chunk helper has no callers anywhere in the
codebase — the streaming path emits proper A2A events via
create_task_event / create_status_update_event /
create_artifact_update_event in handler.py. Removing the dead method
also eliminates the inconsistency where the unused chunk inlined the
envelope-level final flag inside the Message result.
* fix(a2a): scope a2a lazy-feature so it doesn't subsume /v1/a2a/discover
- _lazy_features.py: use /a2a prefix + /message/send suffix for the
a2a feature so a request to /v1/a2a/discover no longer triggers the
a2a_endpoints module to load alongside a2a_registration.
- agent_endpoints/endpoints.py: drop the no-op description override
kwarg from _build_merged_agent_card and its three call sites. The
upstream card's description is already preserved by merge_agent_card's
deepcopy, so passing it explicitly did nothing.
* style: black-format litellm/a2a_protocol/litellm_completion_bridge/transformation.py
* fix: address PR bugfix review for a2a discovery + metadata forwarding
- agent create form (add_agent_form.tsx): drop the skills.length > 0
guard so an admin can clear all discovered skills during creation,
matching the edit form's overlay behavior (consistency between
create and edit flows).
- agent_card_discovery.tsx: stop including savedAgentCard in the
handleDiscover useCallback deps. Read it via a ref inside
resetSelections instead, so a parent-driven re-render that hands us
a new savedAgentCard object reference (e.g. a background refresh of
the agent record) does not recreate handleDiscover and re-fire the
auto-discover effect, which would otherwise overwrite in-progress
user edits in parent-driven mode (debounceMs = 0).
- a2a_endpoints.invoke_agent_a2a: skip 'metadata' when moving
litellm params off of A2A MessageSendParams into body. The A2A
protocol defines params.metadata as a first-class request-level
field, and the completion bridge's get_forward_metadata is supposed
to merge it with message.metadata. Previously the proxy always
stripped params.metadata before constructing MessageSendParams, so
the params-level branch in get_forward_metadata was dead code in
the proxy flow.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(a2a): return 404 from get_agent_card when agent has no card
* fix(agents): apply discovery overlay uniformly on create and dedupe ALLOWED_CAPABILITY_KEYS
- buildAgentData now applies overlayDiscoveredCardParams after every
non-custom branch (a2a, use_a2a_form_fields, dynamic) so types with
credential_fields no longer silently drop discovered skills,
capabilities, input/output modes, provider, and icon/doc URLs on
submit. Mirrors the edit flow in agent_info.tsx.
- Export ALLOWED_CAPABILITY_KEYS from agent_discovery_utils and import
it in agent_card_discovery so the rendering and selection-filtering
logic share a single source of truth.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* ci(proxy-endpoints): wire tests/test_litellm/proxy/a2a into the shard
The two new test files (test_discovery.py, test_agent_card.py) were
not picked up by any pytest path, so their coverage never reached
codecov and patch coverage fell below the auto target.
* fix(ui): overlay discovered name/description in create flow for dynamic agents
Mirror the edit-form overlay in agent_info.tsx so dynamic agent types
(e.g. LangGraph) whose forms don't register name/description as
Form.Items don't silently lose those discovery-panel edits on save.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(a2a): default merged agent card version, null-guard runtime URL lookup, scope discovery auto-fire to A2A types
- merge_agent_card now defaults version to 1.0.0 when upstream omits it
(A2A v1.0 schema requires the field).
- invoke_agent_a2a guards against agent_card_params being None so plain
chat agents routed via the A2A path return a JSON-RPC error instead of
AttributeError.
- buildDiscoveryRequest no longer falls back to any URL-shaped credential
field for non-A2A agent types (Azure AI Foundry, Bedrock AgentCore,
Vertex). Discovery only auto-fires for pure A2A and use_a2a_form_fields
runtimes; the manual URL input remains available as an escape hatch.
* fix(ui): extract overlayDiscoveredCardParams + debounce parent-driven discovery
Two findings from greptile review:
1. `overlayDiscoveredCardParams` was copy-pasted between `add_agent_form.tsx`
and `agent_info.tsx`. Move it to `agent_discovery_utils.ts` so the create
and edit flows share the same overlay logic and there's only one place to
update when discovered fields change.
2. `agent_card_discovery.tsx` used a zero-debounce path for parent-driven
mode, which fires one discovery HTTP request per keystroke when an admin
types into the parent form's URL / api_base / assistant_id fields (the
parent rebuilds the plan from watched form values every render). Apply
the same 400ms debounce uniformly.
* fix(a2a): preserve discovery name edit, default discovery headers, sync url on re-discover
- _build_merged_agent_card: prefer card-supplied name over agent_name so
the discovery panel's editable 'Name (shown to API clients)' value is
not silently overwritten by the internal identifier.
- async_safe_get call in fetch_well_known_card: pass headers or {} to
avoid TypeError({**None, 'Host': ...}) when URL validation is enabled
in production (default).
- agent_info handleApplyDiscoveredCard: set url: selection.upstream_url
in fieldsToSet so re-discovery during edit refreshes the form's URL
field for pure A2A agents (matches add_agent_form).
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(a2a): scrub upstream url from /public/agent_hub cards
Public agent_hub returned agent_card_params verbatim, exposing the
retained upstream backend url to unauthenticated callers. Rewrite the
url to the proxy /a2a/{agent_id} entrypoint on response, matching the
behavior of the authenticated well-known agent-card endpoint, so the
backend cannot be reached outside LiteLLM's auth, budget, and logging
path.
* fix(a2a): include suffix-matched routes in lazy warm openapi fragment
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): add harness for proxy_server.py behavior-pinning
Creates tests/test_litellm/proxy/proxy_server/ with:
- conftest.py: 11 shared fixtures (app, client, mock_prisma, auth_as,
mock_router with parametrized response builders, normalize, etc.)
- _coverage_check.py: per-PR coverage gate (line + branch) against a
baseline, self-selects target by inspecting which placeholder files
have been filled
- _pin_check.py: AST-based gate that verifies every pin-list item has
>=1 happy + >=1 error test with a real assertion (no status-only)
- test_harness_smoke.py: 19 smoke tests covering every fixture +
both scripts end-to-end
- 26 placeholder test files (one docstring each) reserved for
follow-up PRs per the directory ownership in the Notion plan
- .coverage_baseline pinned at 0% so future PRs measure deltas
against new-tests-only and aren't entangled with the broader
scattered test suite
Adds a dedicated proxy-server job to test-unit-proxy-endpoints.yml
so this directory's runtime + coverage are tracked independently.
Plan: https://www.notion.so/36c43b8acdab81ee845fd5365128a2fc
* ci(proxy-endpoints): allow workflow_dispatch
Lets the workflow be triggered manually on a branch via
`gh workflow run`, which is needed for the verify-first
flow on workflow changes before opening a PR.
* test(proxy): address review feedback on proxy_server harness
- conftest.py: anchor sys.path insert to __file__ (Path(__file__).resolve().parents[4])
instead of CWD-relative os.path.abspath("../../../../") which resolved
to the wrong directory when pytest is launched from the repo root.
- _coverage_check.py: actually read .coverage_baseline and use it as
the floor (line_min = max(target, baseline)). Closes the gap between
the PR description's "delta semantics" and what the script was doing.
With baseline=0.0 today this is a no-op; future PRs that update the
baseline cause regressions (test deletions etc.) to trip the gate
even if the static PR target is still met.
- _pin_check.py: drop unreachable startswith("_") guard
(test_*.py glob never yields underscore-prefixed names) and read
each test file once instead of twice.
Split into two related cleanups:
1. Delete CCI jobs that duplicate GHA coverage:
- mcp_testing (tests/mcp_tests) — already run by test-mcp.yml
- litellm_mapped_tests_proxy_part1/part2 (tests/test_litellm/proxy) —
already run across test-unit-proxy-auth.yml, test-unit-proxy-endpoints.yml,
and test-unit-proxy-infra.yml
Add rag_endpoints and realtime_endpoints to test-unit-proxy-endpoints.yml
(they were only covered by the deleted CCI part2 job).
Remove the corresponding workflow wiring, coverage combine entries, and
upload-coverage dependencies in .circleci/config.yml.
2. Re-shard test-unit-proxy-db.yml from 4 alphabetic buckets to 8 semantic
ones (auth-and-jwt, proxy-server, logging-and-callbacks, db-and-spend,
guardrails-budget-hooks, endpoints-and-responses, plus the existing
serial key-generation and test_proxy_utils.py shards). New test files are
placed in whichever group they belong to instead of reshuffling slices.
Add a dist input to _test-unit-services-base.yml so the test_proxy_utils.py
shard can use --dist=worksteal to spread its ~64 (many parametrized)
functions across workers; the default --dist=loadscope pins a single file
to a single worker, which was the root cause of that shard running 10m+.
Required test-unit-* and related workflows only triggered on PRs targeting
main, so feature PRs routed through litellm_internal_staging or
litellm_oss_branch never dispatched the full suite. Branch protection
reported BLOCKED even when CircleCI was green.
Expand pull_request and push branch filters to also match
litellm_internal_staging, litellm_oss_branch, and "litellm_**" (using **
so branch names containing "/" also match).
* Remove redundant matrix unit test workflow
All test paths in test-litellm-matrix.yml are fully covered by the
newer semantic unit test workflows (test-unit-*.yml), making the
matrix workflow redundant CI spend.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add Codecov coverage reporting to semantic unit test workflows
Add coverage collection (--cov) and Codecov OIDC upload to both
reusable base workflows and all 12 caller workflows, replacing the
coverage reporting that was previously only in the matrix workflow.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Move id-token/pull-requests permissions to job level for multi-job workflows
For workflows with multiple jobs (llm-providers, proxy-db), move
id-token: write and pull-requests: write from workflow level to job
level so permissions are scoped to only the jobs that need them.
Removes zizmor inline suppressions that were masking the issue.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rename job keys from generic 'test' to descriptive names (e.g.,
'core-utils', 'proxy-auth', 'router') so GitHub checks display as
'core-utils / run' instead of 'test / test'.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace monolithic matrix workflow with individual, descriptively-named
workflow files. Each workflow uses a shared reusable base and follows
least-privilege security: zero secrets, read-only permissions, SHA-pinned
actions, persist-credentials: false, and env-var indirection to prevent
template injection.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
