Contextual CVSS as a full 8-metric breakdown, computed like a normal finding

This commit is contained in:
Ahmed Allam 2026-08-17 09:45:04 +00:00 committed by Ahmed Allam
parent 9c0d30a0d0
commit e442db9c93
4 changed files with 256 additions and 135 deletions

View file

@ -215,14 +215,11 @@ def render_vulnerability_md(report: dict[str, Any]) -> str: # noqa: PLR0912, PL
cvss = report.get("cvss") cvss = report.get("cvss")
if cvss is not None: if cvss is not None:
metadata.append(("CVSS", cvss)) metadata.append(("CVSS", cvss))
contextual_metrics = dep_meta.get("contextual_cvss_metrics") advisory_cvss = dep_meta.get("advisory_cvss")
if isinstance(contextual_metrics, dict) and contextual_metrics: if advisory_cvss is not None and advisory_cvss != cvss:
metadata.append( metadata.append(("Advisory CVSS", advisory_cvss))
( if dep_meta.get("contextual_cvss_vector"):
"Contextual CVSS Metrics", metadata.append(("Contextual CVSS Vector", dep_meta["contextual_cvss_vector"]))
"/".join(f"{metric}:{value}" for metric, value in contextual_metrics.items()),
)
)
if report.get("fix_effort"): if report.get("fix_effort"):
metadata.append(("Fix Effort", str(report["fix_effort"]).title())) metadata.append(("Fix Effort", str(report["fix_effort"]).title()))
for label, value in metadata: for label, value in metadata:

View file

@ -241,63 +241,68 @@ findings and rejects empty PoC fields):
installed/affected version, fixed version, lockfile path, and the relevant installed/affected version, fixed version, lockfile path, and the relevant
trivy output excerpt. trivy output excerpt.
- **Always set `advisory_cvss` to the published advisory base score (0.010.0).** - **Always set `advisory_cvss` to the published advisory base score (0.010.0).**
Severity is derived *solely* from this number: read it off the advisory (`CVSS` It is the published reference, and it rates the finding whenever you give no
in trivy output, or the NVD/GHSA page) and pass the real value. The tool rejects contextual breakdown: read it off the advisory (`CVSS` in trivy output, or the
a call that omits it, because guessing a score both inflates low CVEs and NVD/GHSA page) and pass the real value. The tool rejects a call that omits it,
deflates critical ones. because guessing a score both inflates low CVEs and deflates critical ones.
- Set `cwe` to the most specific `CWE-NNN` when the advisory names one. - Set `cwe` to the most specific `CWE-NNN` when the advisory names one.
- Do NOT cap severity at LOW just because there is no dynamic reproduction — use - Do NOT cap severity at LOW just because there is no dynamic reproduction — use
the advisory score. the advisory score.
- Set `reachability` + `reachability_evidence` from the usage analysis above; - Set `reachability` + `reachability_evidence` from the usage analysis above;
use `assumptions` for anything softer (confidence, caveats, analysis limits). use `assumptions` for anything softer (confidence, caveats, analysis limits).
- Set `contextual_cvss_metrics` + `contextual_cvss_reasoning` when this codebase - Set `contextual_cvss_breakdown` + `contextual_cvss_reasoning` when this
clearly changes the risk the published score describes (see below). codebase clearly changes the risk the published score describes (see below).
### Contextual (environmental) CVSS ### Contextual CVSS
The published score rates the CVE in the abstract. `contextual_cvss_metrics` The published score rates the CVE in the abstract. `contextual_cvss_breakdown`
rates it **here**, in this codebase, with CVSS v3.1 environmental metrics. The rates it **here**, in this codebase — the same 8-metric CVSS v3.1 object as a
advisory's base metrics stay fixed — you never restate them and never pass a normal finding's `cvss_breakdown` (`attack_vector`, `attack_complexity`,
score, the adjusted score is computed from the resulting vector. `privileges_required`, `user_interaction`, `scope`, `confidentiality`,
`integrity`, `availability`). You never pass a score: the contextual score and
vector are computed from the breakdown, and when you provide one it determines
the finding's severity. `advisory_cvss` stays the published reference.
Set only what your usage analysis supports: Start from the advisory's own published metrics and change only what your
evidence proves is different in this codebase:
- `MAV` `N`/`A`/`L`/`P` the attack vector as deployed. A library reached only - `attack_vector` `N`/`A`/`L`/`P` — as deployed. A library reached only by a
by a local CLI is `L`, not `N`. local CLI is `L`, not `N`.
- `MAC` `L`/`H` — raise to `H` when the vulnerable path needs a precondition the - `attack_complexity` `L`/`H` — raise to `H` when the vulnerable path needs a
code enforces (input validation, a non-default flag, an internal-only route). precondition the code enforces (input validation, a non-default flag, an
- `MPR` `N`/`L`/`H`, `MUI` `N`/`R` — privileges or interaction this deployment internal-only route).
requires before the path is reachable. - `privileges_required` `N`/`L`/`H`, `user_interaction` `N`/`R` — what this
- `MS` `U`/`C` — whether exploitation here escapes the component boundary. deployment requires before the path is reachable.
- `MC`/`MI`/`MA` `H`/`L`/`N` — the impact in this codebase. `not_imported` code - `scope` `U`/`C` — whether exploitation here escapes the component boundary.
the build still ships is usually `N` across all three. - `confidentiality`/`integrity`/`availability` `N`/`L`/`H` — the impact in this
- `CR`/`IR`/`AR` `H`/`M`/`L` — the security requirement of the data or service codebase. `not_imported` code the build still ships is usually `N` across all
the package handles (credentials or payment data raise `CR`). three.
Ground every metric in the **source-to-sink trace** from the usage analysis Ground every metric in the **source-to-sink trace** from the usage analysis
(step 3 above), not in a general impression of the package. Derive the metrics (step 3 above), not in a general impression of the package. Derive the metrics
from that chain: `MAV`, `MPR`, and `MUI` come from what the source requires; from that chain: `attack_vector`, `privileges_required`, and `user_interaction`
`MAC` comes from the preconditions the hops enforce; `MC`, `MI`, and `MA` come come from what the source requires; `attack_complexity` comes from the
from the data and privileges available at the sink; `CR`, `IR`, and `AR` come preconditions the hops enforce; `confidentiality`, `integrity`, and
from what that data is worth. `availability` come from the data and privileges available at the sink.
No trace, no contextual metrics: if you did not reach a symbol hit, or you No trace, no contextual breakdown: if you did not reach a symbol hit, or you
could not follow a hop, omit the contextual fields instead of guessing. could not follow a hop, omit the contextual fields instead of guessing.
`contextual_cvss_reasoning` is required with the metrics. Write two to four `contextual_cvss_reasoning` is required with the breakdown. Write two to four
sentences that another engineer can check without opening the repository. Name sentences that another engineer can check without opening the repository. Name
the chain hop by hop as `entry point -> intermediate call -> package call`, with the chain hop by hop as `entry point -> intermediate call -> package call`, with
a repository-relative `file:line` for each hop, say who controls the input, and a repository-relative `file:line` for each hop, say who controls the input, and
say what the adjustment changes. Example: `{"MAC": "H", "MC": "L"}` with "The say what the contextual rating changes. Example: lowering `attack_vector` to
only caller of `yaml.load` is `parse_manifest` in `scripts/import.py:88`, which `L` and `confidentiality` to `L` with "The only caller of `yaml.load` is
`cli/commands.py:212` invokes for an operator-supplied path behind the `parse_manifest` in `scripts/import.py:88`, which `cli/commands.py:212` invokes
`--allow-unsafe-import` flag that `deploy/prod.yaml` never sets. No HTTP route for an operator-supplied path behind the `--allow-unsafe-import` flag that
reaches that function, so an attacker must already hold shell access on the job `deploy/prod.yaml` never sets. No HTTP route reaches that function, so an
host, and the parsed data is build metadata rather than customer records." attacker must already hold shell access on the job host, and the parsed data is
build metadata rather than customer records."
Omit all the contextual fields when the published rating already fits, and when Omit all the contextual fields when the published rating already fits, and when
the evidence is thin. A contextual score is a claim you must be able to defend, the evidence is thin. A contextual rating is a claim you must be able to
and this adjustment never replaces `advisory_cvss`. defend, and it never replaces `advisory_cvss` as the published reference.
Verify the CVE with `web_search` when available before reporting. Never guess or Verify the CVE with `web_search` when available before reporting. Never guess or
hallucinate a CVE id. hallucinate a CVE id.
@ -308,12 +313,12 @@ hallucinate a CVE id.
`create_dependency_report`. `create_dependency_report`.
- Do not report a finding without a verified CVE id. - Do not report a finding without a verified CVE id.
- Do not batch multiple CVEs into one report. - Do not batch multiple CVEs into one report.
- Do not omit `advisory_cvss` — the tool rejects it, and it is the single input - Do not omit `advisory_cvss` — the tool rejects it, and it rates every finding
that determines dependency severity. that carries no contextual breakdown.
- Do not silently drop a known CVE because it lacks a dynamic PoC — that is the - Do not silently drop a known CVE because it lacks a dynamic PoC — that is the
exact failure this skill prevents. exact failure this skill prevents.
- Do not downgrade advisory severity for lack of dynamic reproduction. - Do not downgrade advisory severity for lack of dynamic reproduction.
- Do not claim a `reachability` level the evidence does not prove — `unknown` - Do not claim a `reachability` level the evidence does not prove — `unknown`
with a reason is always acceptable; an overclaimed level never is. with a reason is always acceptable; an overclaimed level never is.
- Do not send `contextual_cvss_metrics` without evidence-backed reasoning, and - Do not send `contextual_cvss_breakdown` without evidence-backed reasoning, and
do not use it to quietly de-rate a CVE you simply could not analyze. do not use it to quietly de-rate a CVE you simply could not analyze.

View file

@ -749,37 +749,59 @@ def _validate_manifest_path(manifest_path: str | None) -> str | None:
return None return None
# CVSS v3.1 environmental metrics an agent may set on a dependency finding,
# with their legal values. The base metrics are deliberately absent: they come
# from the published advisory, so a report can never restate them.
_CVSS_ENVIRONMENTAL_VALUES: dict[str, frozenset[str]] = {
"MAV": frozenset("NALP"),
"MAC": frozenset("LH"),
"MPR": frozenset("NLH"),
"MUI": frozenset("NR"),
"MS": frozenset("UC"),
"MC": frozenset("HLN"),
"MI": frozenset("HLN"),
"MA": frozenset("HLN"),
"CR": frozenset("HML"),
"IR": frozenset("HML"),
"AR": frozenset("HML"),
}
_MAX_CONTEXTUAL_REASONING_CHARS = 2000 _MAX_CONTEXTUAL_REASONING_CHARS = 2000
def _clean_contextual_cvss_metrics(raw: dict[str, str] | None) -> dict[str, str]: def _validate_contextual_cvss(
"""Keep only well-formed CVSS environmental metrics from a report.""" breakdown: dict[str, str] | None,
if not isinstance(raw, dict): reasoning: str | None,
return {} ) -> list[str]:
metrics: dict[str, str] = {} errors: list[str] = []
for key, value in raw.items(): if breakdown:
metric = str(key or "").strip().upper() for name, valid in _CVSS_VALID.items():
metric_value = str(value or "").strip().upper() value = breakdown.get(name)
allowed = _CVSS_ENVIRONMENTAL_VALUES.get(metric) if value not in valid:
if allowed and metric_value in allowed: errors.append(
metrics[metric] = metric_value f"Invalid contextual_cvss_breakdown {name}: {value}. Must be one of: {valid}"
return metrics )
if not (reasoning or "").strip():
errors.append(
"contextual_cvss_reasoning is required when contextual_cvss_breakdown is "
"set: state what you observed in this codebase that justifies the "
"contextual rating. A contextual score with no reasoning is not shown."
)
return errors
def _validate_advisory_cvss(advisory_cvss: float | None) -> str | None:
if advisory_cvss is None:
return (
"advisory_cvss is required: read the published advisory base score "
"(0.0-10.0) off the advisory (trivy CVSS / NVD / GHSA). It is the "
"published reference the finding is rated against — do not omit it "
"or the finding cannot be rated."
)
if not 0.0 <= advisory_cvss <= 10.0:
return f"advisory_cvss must be between 0.0 and 10.0, got {advisory_cvss}"
return None
def _resolve_dependency_rating(
advisory_cvss: float | None,
contextual_cvss_breakdown: dict[str, str] | None,
) -> tuple[float | None, str, float | None, str | None]:
"""Rate the finding.
A contextual breakdown works exactly like a normal finding's
``cvss_breakdown``: the agent supplies the 8 metrics as observed in this
codebase and the score/vector are computed from them. When provided it
rates the finding; the advisory score stays as the published reference.
"""
if contextual_cvss_breakdown:
score, severity, vector = _calculate_cvss(contextual_cvss_breakdown)
return score, severity, score, vector
score, severity = _dependency_severity(advisory_cvss)
return score, severity, None, None
def _build_dependency_metadata( def _build_dependency_metadata(
@ -793,13 +815,18 @@ def _build_dependency_metadata(
manifest_path: str | None = None, manifest_path: str | None = None,
reachability: str | None = None, reachability: str | None = None,
reachability_evidence: str | None = None, reachability_evidence: str | None = None,
contextual_cvss_metrics: dict[str, str] | None = None, advisory_cvss: float | None = None,
contextual_cvss_breakdown: dict[str, str] | None = None,
contextual_cvss_score: float | None = None,
contextual_cvss_vector: str | None = None,
contextual_cvss_reasoning: str | None = None, contextual_cvss_reasoning: str | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
metadata: dict[str, Any] = { metadata: dict[str, Any] = {
"package_name": package_name.strip(), "package_name": package_name.strip(),
"installed_version": installed_version.strip(), "installed_version": installed_version.strip(),
} }
if advisory_cvss is not None:
metadata["advisory_cvss"] = advisory_cvss
if package_ecosystem and package_ecosystem.strip(): if package_ecosystem and package_ecosystem.strip():
metadata["package_ecosystem"] = package_ecosystem.strip() metadata["package_ecosystem"] = package_ecosystem.strip()
if manifest_path and manifest_path.strip(): if manifest_path and manifest_path.strip():
@ -816,12 +843,19 @@ def _build_dependency_metadata(
metadata["reachability"] = reachability.strip() metadata["reachability"] = reachability.strip()
if reachability_evidence and reachability_evidence.strip(): if reachability_evidence and reachability_evidence.strip():
metadata["reachability_evidence"] = reachability_evidence.strip() metadata["reachability_evidence"] = reachability_evidence.strip()
# Contextual CVSS is only meaningful as metrics plus the reasoning a reader # Contextual CVSS is only meaningful as the full breakdown, its computed
# can check, so an incomplete pair is dropped. # score/vector, and the reasoning a reader can check — an incomplete set
cleaned_metrics = _clean_contextual_cvss_metrics(contextual_cvss_metrics) # is dropped.
reasoning = str(contextual_cvss_reasoning or "").strip() reasoning = str(contextual_cvss_reasoning or "").strip()
if cleaned_metrics and reasoning: if (
metadata["contextual_cvss_metrics"] = cleaned_metrics contextual_cvss_breakdown
and contextual_cvss_score is not None
and contextual_cvss_vector
and reasoning
):
metadata["contextual_cvss_breakdown"] = contextual_cvss_breakdown
metadata["contextual_cvss_score"] = contextual_cvss_score
metadata["contextual_cvss_vector"] = contextual_cvss_vector
metadata["contextual_cvss_reasoning"] = reasoning[:_MAX_CONTEXTUAL_REASONING_CHARS] metadata["contextual_cvss_reasoning"] = reasoning[:_MAX_CONTEXTUAL_REASONING_CHARS]
return metadata return metadata
@ -894,7 +928,7 @@ async def _do_create_dependency( # noqa: PLR0912
manifest_path: str | None = None, manifest_path: str | None = None,
reachability: str = "unknown", reachability: str = "unknown",
reachability_evidence: str | None = None, reachability_evidence: str | None = None,
contextual_cvss_metrics: dict[str, str] | None = None, contextual_cvss_breakdown: dict[str, str] | None = None,
contextual_cvss_reasoning: str | None = None, contextual_cvss_reasoning: str | None = None,
agent_id: str | None = None, agent_id: str | None = None,
agent_name: str | None = None, agent_name: str | None = None,
@ -948,28 +982,21 @@ async def _do_create_dependency( # noqa: PLR0912
"govulncheck call path). Never claim a reachability level without evidence." "govulncheck call path). Never claim a reachability level without evidence."
) )
if _clean_contextual_cvss_metrics(contextual_cvss_metrics) and not ( errors.extend(_validate_contextual_cvss(contextual_cvss_breakdown, contextual_cvss_reasoning))
contextual_cvss_reasoning or ""
).strip():
errors.append(
"contextual_cvss_reasoning is required when contextual_cvss_metrics is set: "
"state in one or two sentences what you observed in this codebase that "
"justifies the adjustment. An adjusted score with no reasoning is not shown."
)
if advisory_cvss is None: advisory_err = _validate_advisory_cvss(advisory_cvss)
errors.append( if advisory_err:
"advisory_cvss is required: read the published advisory base score " errors.append(advisory_err)
"(0.0-10.0) off the advisory (trivy CVSS / NVD / GHSA). Severity is "
"derived solely from it — do not omit it or the finding cannot be rated."
)
elif not 0.0 <= advisory_cvss <= 10.0:
errors.append(f"advisory_cvss must be between 0.0 and 10.0, got {advisory_cvss}")
if errors: if errors:
return {"success": False, "error": "Validation failed", "errors": errors} return {"success": False, "error": "Validation failed", "errors": errors}
cvss_score, severity = _dependency_severity(advisory_cvss) try:
cvss_score, severity, contextual_score, contextual_vector = _resolve_dependency_rating(
advisory_cvss, contextual_cvss_breakdown
)
except ValueError as exc:
return {"success": False, "error": "Validation failed", "errors": [str(exc)]}
dependency_metadata = _build_dependency_metadata( dependency_metadata = _build_dependency_metadata(
package_name=package_name, package_name=package_name,
installed_version=installed_version, installed_version=installed_version,
@ -980,7 +1007,10 @@ async def _do_create_dependency( # noqa: PLR0912
manifest_path=manifest_path, manifest_path=manifest_path,
reachability=reachability, reachability=reachability,
reachability_evidence=reachability_evidence, reachability_evidence=reachability_evidence,
contextual_cvss_metrics=contextual_cvss_metrics, advisory_cvss=advisory_cvss,
contextual_cvss_breakdown=contextual_cvss_breakdown,
contextual_cvss_score=contextual_score,
contextual_cvss_vector=contextual_vector,
contextual_cvss_reasoning=contextual_cvss_reasoning, contextual_cvss_reasoning=contextual_cvss_reasoning,
) )
evidence = _build_dependency_evidence( evidence = _build_dependency_evidence(
@ -1093,7 +1123,7 @@ async def create_dependency_report(
dependency_path: str | None = None, dependency_path: str | None = None,
reachability: str = "unknown", reachability: str = "unknown",
reachability_evidence: str | None = None, reachability_evidence: str | None = None,
contextual_cvss_metrics: dict[str, str] | None = None, contextual_cvss_breakdown: dict[str, str] | None = None,
contextual_cvss_reasoning: str | None = None, contextual_cvss_reasoning: str | None = None,
) -> str: ) -> str:
"""File a known-CVE dependency (SCA) finding — one report per CVE x package. """File a known-CVE dependency (SCA) finding — one report per CVE x package.
@ -1137,8 +1167,10 @@ async def create_dependency_report(
proved a path from application code to the vulnerable function. proved a path from application code to the vulnerable function.
- ``unknown`` usage analysis was not performed or was inconclusive. - ``unknown`` usage analysis was not performed or was inconclusive.
Severity is still derived solely from ``advisory_cvss`` the Severity comes from ``contextual_cvss_breakdown`` when you provide one
reachability level never changes the rating, only prioritization. (computed exactly like a normal finding's ``cvss_breakdown``), otherwise
from ``advisory_cvss``. The reachability level alone never changes the
rating, only prioritization.
**Formatting**: use markdown in text fields (``**bold**``, ``inline **Formatting**: use markdown in text fields (``**bold**``, ``inline
code`` for package/version identifiers, fenced code blocks for code`` for package/version identifiers, fenced code blocks for
@ -1159,8 +1191,9 @@ async def create_dependency_report(
cwe: ``CWE-NNN`` (most specific) if certain, else omit. cwe: ``CWE-NNN`` (most specific) if certain, else omit.
advisory_cvss: **Required.** Published advisory base score advisory_cvss: **Required.** Published advisory base score
(0.0-10.0) read it off the advisory (trivy CVSS / NVD / GHSA). (0.0-10.0) read it off the advisory (trivy CVSS / NVD / GHSA).
Severity is derived solely from this score, so it must be the It is the published reference the finding is rated against and
real published value; do not guess or omit it. rates the finding whenever you give no contextual breakdown, so
it must be the real published value; do not guess or omit it.
technical_analysis: Optional deeper mechanism/root-cause detail. technical_analysis: Optional deeper mechanism/root-cause detail.
fix_effort: One of ``trivial`` / ``low`` / ``medium`` / ``high`` fix_effort: One of ``trivial`` / ``low`` / ``medium`` / ``high``
(dependency upgrades are usually ``trivial``/``low``). (dependency upgrades are usually ``trivial``/``low``).
@ -1199,26 +1232,29 @@ async def create_dependency_report(
is off in production), and say who controls the input. State is off in production), and say who controls the input. State
it plainly when no entry point reaches the sink that is the it plainly when no entry point reaches the sink that is the
most useful result a reader can get. most useful result a reader can get.
contextual_cvss_metrics: Optional CVSS v3.1 **environmental** contextual_cvss_breakdown: Optional full CVSS v3.1 rating of this
metrics that reframe the published score for this codebase, CVE **in this codebase** the same 8-metric object as
as a mapping of metric to value: ``MAV`` (N/A/L/P), ``MAC`` ``create_vulnerability_report``'s ``cvss_breakdown``:
(L/H), ``MPR`` (N/L/H), ``MUI`` (N/R), ``MS`` (U/C), ``MC`` / ``attack_vector`` (N/A/L/P), ``attack_complexity`` (L/H),
``MI`` / ``MA`` (H/L/N), ``CR`` / ``IR`` / ``AR`` (H/M/L). ``privileges_required`` (N/L/H), ``user_interaction`` (N/R),
Set only the metrics your evidence supports (for example ``scope`` (U/C), ``confidentiality`` / ``integrity`` /
``{"MAC": "H", "MC": "L"}`` when the vulnerable path needs a ``availability`` (N/L/H). All 8 metrics are required when the
precondition this deployment enforces and the data at risk is field is set, and the contextual score/vector are computed
limited). Derive every value from the **source-to-sink from them you never supply a score. Start from the
trace** you recorded in ``reachability_evidence``: adjust advisory's published metrics and change only what the
``MAV`` / ``MPR`` / ``MUI`` from what the entry point **source-to-sink trace** you recorded in
actually requires, ``MAC`` from the preconditions the hops ``reachability_evidence`` proves is different here: derive
enforce, and ``MC`` / ``MI`` / ``MA`` from the data and ``attack_vector`` / ``privileges_required`` /
privileges reachable at the sink. You never supply base ``user_interaction`` from what the entry point actually
metrics or a score: the base vector comes from the advisory requires, ``attack_complexity`` from the preconditions the
and the adjusted score is computed from the resulting vector. hops enforce, and the impact metrics from the data and
Omit the field when the trace does not change the published privileges reachable at the sink. When provided, this rating
rating, or when you could not complete the trace. determines the finding's severity; ``advisory_cvss`` stays as
the published reference. Omit the field when the trace does
not change the published rating, or when you could not
complete the trace.
contextual_cvss_reasoning: **Required whenever** contextual_cvss_reasoning: **Required whenever**
``contextual_cvss_metrics`` is set. Two to four detailed ``contextual_cvss_breakdown`` is set. Two to four detailed
sentences that a reviewer can verify without opening the repo: sentences that a reviewer can verify without opening the repo:
how the application uses the package, which call sites or how the application uses the package, which call sites or
configuration you inspected (repo-relative ``file:line``), configuration you inspected (repo-relative ``file:line``),
@ -1254,7 +1290,7 @@ async def create_dependency_report(
manifest_path=manifest_path, manifest_path=manifest_path,
reachability=reachability, reachability=reachability,
reachability_evidence=reachability_evidence, reachability_evidence=reachability_evidence,
contextual_cvss_metrics=contextual_cvss_metrics, contextual_cvss_breakdown=contextual_cvss_breakdown,
contextual_cvss_reasoning=contextual_cvss_reasoning, contextual_cvss_reasoning=contextual_cvss_reasoning,
agent_id=agent_id, agent_id=agent_id,
agent_name=agent_name, agent_name=agent_name,

View file

@ -160,6 +160,7 @@ async def test_dependency_report_sets_class_and_metadata(report_state: ReportSta
assert report["dependency_metadata"] == { assert report["dependency_metadata"] == {
"package_name": "lodash", "package_name": "lodash",
"installed_version": "4.17.20", "installed_version": "4.17.20",
"advisory_cvss": 7.2,
"package_ecosystem": "npm", "package_ecosystem": "npm",
"manifest_path": "package-lock.json", "manifest_path": "package-lock.json",
"fixed_version": "4.17.21", "fixed_version": "4.17.21",
@ -463,6 +464,7 @@ async def test_dependency_report_dedupe_candidate_includes_dependency_metadata(
"dependency_metadata": { "dependency_metadata": {
"package_name": "sample", "package_name": "sample",
"installed_version": "1.0.0", "installed_version": "1.0.0",
"advisory_cvss": 0.0,
"package_ecosystem": "npm", "package_ecosystem": "npm",
"manifest_path": "package-lock.json", "manifest_path": "package-lock.json",
"fixed_version": "1.0.1", "fixed_version": "1.0.1",
@ -882,17 +884,29 @@ def test_vuln_tool_exposes_new_params() -> None:
def test_dep_tool_exposes_contextual_cvss_params() -> None: def test_dep_tool_exposes_contextual_cvss_params() -> None:
dep_props = create_dependency_report.params_json_schema["properties"] dep_props = create_dependency_report.params_json_schema["properties"]
for field in ( for field in (
"contextual_cvss_metrics", "contextual_cvss_breakdown",
"contextual_cvss_reasoning", "contextual_cvss_reasoning",
): ):
assert field in dep_props assert field in dep_props
assert "source-to-sink" in dep_props["contextual_cvss_metrics"]["description"].lower() assert "source-to-sink" in dep_props["contextual_cvss_breakdown"]["description"].lower()
assert "source-to-sink" in dep_props["reachability_evidence"]["description"].lower() assert "source-to-sink" in dep_props["reachability_evidence"]["description"].lower()
assert "file:line" in dep_props["contextual_cvss_reasoning"]["description"].lower() assert "file:line" in dep_props["contextual_cvss_reasoning"]["description"].lower()
_CONTEXTUAL_BREAKDOWN = {
"attack_vector": "L",
"attack_complexity": "H",
"privileges_required": "H",
"user_interaction": "N",
"scope": "U",
"confidentiality": "L",
"integrity": "L",
"availability": "N",
}
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_dependency_report_keeps_only_valid_contextual_metrics( async def test_dependency_report_computes_contextual_cvss(
report_state: ReportState, report_state: ReportState,
) -> None: ) -> None:
result = await _do_create_dependency( result = await _do_create_dependency(
@ -912,18 +926,25 @@ async def test_dependency_report_keeps_only_valid_contextual_metrics(
cwe="CWE-94", cwe="CWE-94",
fix_effort="trivial", fix_effort="trivial",
manifest_path="package-lock.json", manifest_path="package-lock.json",
contextual_cvss_metrics={"MAC": "H", "MC": "L", "AV": "N", "MPR": "Z"}, contextual_cvss_breakdown=_CONTEXTUAL_BREAKDOWN,
contextual_cvss_reasoning="Only scripts/import.py reaches the sink.", contextual_cvss_reasoning="Only scripts/import.py reaches the sink.",
) )
assert result["success"] is True, result assert result["success"] is True, result
metadata = report_state.vulnerability_reports[0]["dependency_metadata"] report = report_state.vulnerability_reports[0]
assert metadata["contextual_cvss_metrics"] == {"MAC": "H", "MC": "L"} metadata = report["dependency_metadata"]
assert metadata["advisory_cvss"] == 7.2
assert metadata["contextual_cvss_breakdown"] == _CONTEXTUAL_BREAKDOWN
assert metadata["contextual_cvss_vector"] == ("CVSS:3.1/AV:L/AC:H/PR:H/UI:N/S:U/C:L/I:L/A:N")
assert metadata["contextual_cvss_score"] == pytest.approx(3.0, abs=0.05)
assert metadata["contextual_cvss_reasoning"] == "Only scripts/import.py reaches the sink." assert metadata["contextual_cvss_reasoning"] == "Only scripts/import.py reaches the sink."
assert "contextual_cvss_metric_reasoning" not in metadata # The contextual rating determines the finding's score/severity, exactly
# like a normal finding's cvss_breakdown.
assert report["cvss"] == metadata["contextual_cvss_score"]
assert report["severity"] == "low"
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_dependency_report_rejects_contextual_metrics_without_reasoning( async def test_dependency_report_rates_from_advisory_without_contextual(
report_state: ReportState, report_state: ReportState,
) -> None: ) -> None:
result = await _do_create_dependency( result = await _do_create_dependency(
@ -943,7 +964,69 @@ async def test_dependency_report_rejects_contextual_metrics_without_reasoning(
cwe="CWE-94", cwe="CWE-94",
fix_effort="trivial", fix_effort="trivial",
manifest_path="package-lock.json", manifest_path="package-lock.json",
contextual_cvss_metrics={"MAC": "H"}, )
assert result["success"] is True, result
report = report_state.vulnerability_reports[0]
assert report["cvss"] == 7.2
assert report["severity"] == "high"
metadata = report["dependency_metadata"]
assert metadata["advisory_cvss"] == 7.2
assert "contextual_cvss_breakdown" not in metadata
assert "contextual_cvss_score" not in metadata
@pytest.mark.asyncio
async def test_dependency_report_rejects_incomplete_contextual_breakdown(
report_state: ReportState,
) -> None:
result = await _do_create_dependency(
title="CVE-2021-23337 in lodash 4.17.20",
description="Command injection via template.",
target="repo/package.json",
cve="CVE-2021-23337",
package_name="lodash",
installed_version="4.17.20",
impact="Arbitrary command execution.",
remediation_steps="Upgrade to 4.17.21.",
assumptions="Assumes the template sink is reachable.",
package_ecosystem="npm",
advisory_cvss=7.2,
technical_analysis=None,
fixed_version="4.17.21",
cwe="CWE-94",
fix_effort="trivial",
manifest_path="package-lock.json",
contextual_cvss_breakdown={"attack_vector": "L", "attack_complexity": "Z"},
contextual_cvss_reasoning="Only scripts/import.py reaches the sink.",
)
assert result["success"] is False
assert any("attack_complexity" in error for error in result["errors"])
assert any("privileges_required" in error for error in result["errors"])
assert report_state.vulnerability_reports == []
@pytest.mark.asyncio
async def test_dependency_report_rejects_contextual_breakdown_without_reasoning(
report_state: ReportState,
) -> None:
result = await _do_create_dependency(
title="CVE-2021-23337 in lodash 4.17.20",
description="Command injection via template.",
target="repo/package.json",
cve="CVE-2021-23337",
package_name="lodash",
installed_version="4.17.20",
impact="Arbitrary command execution.",
remediation_steps="Upgrade to 4.17.21.",
assumptions="Assumes the template sink is reachable.",
package_ecosystem="npm",
advisory_cvss=7.2,
technical_analysis=None,
fixed_version="4.17.21",
cwe="CWE-94",
fix_effort="trivial",
manifest_path="package-lock.json",
contextual_cvss_breakdown=_CONTEXTUAL_BREAKDOWN,
contextual_cvss_reasoning=" ", contextual_cvss_reasoning=" ",
) )
assert result["success"] is False assert result["success"] is False