Skip to content

Commit 44ff6cc

Browse files
cwayne18claude
andcommitted
Add --severity to limit the report to the ratings worth acting on
A debian:12 --all scan is 161 findings, and the only way to get the subset worth waking someone for was jq over --format json. This is Trivy's spelling: --severity CRITICAL,HIGH, comma-separated or repeatable, case-insensitive, MODERATE accepted for MEDIUM. The flag is nearly free -- stringList already splits on commas. All the work is in what a filter is allowed to hide, because this tool has one rule it never breaks: a short report must never read as a clean one. So every filtered run says what it withheld, and glosses the unrated count. UNKNOWN is a severity you have to name, as in Trivy, and on debian:12 that hides 36 findings whose only crime is a CVSS v4-only record -- "36 unknown (no rating was published)" is the line that keeps that honest. In --repo mode it is worse: govulncheck's OpenVEX carries no severity, so any --severity that omits UNKNOWN filters out everything, and that case prints "This is a filtered view, not a clean result" rather than the ordinary no-findings line. One exemption: a --cves id that matched no component always survives. Those rows exist so a hand-typed id cannot vanish; deleting them would recreate exactly the silence they were written to prevent. Parsing is strict, via a new cvss.Parse rather than cvss.Normalize -- Normalize("CRITCAL") is UNKNOWN, so the lenient version would read a typo as a request for precisely the unrated findings. A bad name is exit 2 before the pull. The filter runs in the orchestrator, not the renderer, so --format json shrinks the same way and gains a matching withheld block, and the LLM overlay is never billed for a row nobody will read. Exit codes are unchanged: findings existing is still not a failure. Measured: debian:12 keeps 38 of 161 at CRITICAL,HIGH with the JSON withheld block agreeing exactly; --repo prints the filtered-view guard at exit 0; and on rancher/hardened-kubernetes --severity composes with --vexhub without costing any of its 3 hub matches. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 7022855 commit 44ff6cc

9 files changed

Lines changed: 658 additions & 19 deletions

File tree

README.md

Lines changed: 57 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -835,13 +835,62 @@ Two things report `UNKNOWN` that are worth knowing about:
835835
- **CVSS 4.0-only records are not scored.** A v4 base score is a 270-entry
836836
MacroVector lookup with interpolation, not a formula. Records carrying only a
837837
v4 vector report `UNKNOWN` rather than a number this tool made up. Most
838-
advisories still publish v3 alongside; on `debian:12` 34 of 159 findings are
838+
advisories still publish v3 alongside; on `debian:12` 36 of 161 findings are
839839
unrated, from a mix of v4-only and pre-CVSS records.
840840
- **`--repo` Go findings carry no severity at all.** That path resolves
841841
advisories inside govulncheck, which is run with `-format openvex`, and OpenVEX
842842
carries no severity field. Image mode goes entirely through the resolver and is
843843
fully covered — on `debian:12 --all` every finding gets a rating.
844844

845+
### Filtering by severity (`--severity`)
846+
847+
`--severity CRITICAL,HIGH` reports only the findings at those ratings. It is
848+
comma-separated or repeatable, case-insensitive, accepts `MODERATE` for
849+
`MEDIUM`, and a name it does not recognize is a command-line error (exit 2)
850+
rather than a silently empty report.
851+
852+
```console
853+
$ vexscan --image debian:12 --all --ecosystem os --severity CRITICAL,HIGH
854+
vexscan report (image) for debian:12
855+
NOTE: --severity CRITICAL,HIGH withheld 123 of 161 findings:
856+
36 unknown (no rating was published), 78 medium, 9 low
857+
858+
os Debian:12 88 components 38 findings
859+
affected by severity: 10 critical, 26 high
860+
```
861+
862+
The filter is applied to the result, not to the rendering, so `--format json`
863+
shrinks the same way and gains a `withheld` block that matches the banner
864+
exactly. It also runs before the LLM overlay, so `--severity CRITICAL --llm`
865+
only pays for criticals.
866+
867+
Three things about it are worth knowing before you put it in CI:
868+
869+
- **`UNKNOWN` is a severity you have to ask for.** As in Trivy, a `--severity`
870+
that does not name it drops it — 36 findings on `debian:12` above. Those are
871+
unrated, not unimportant ([above](#severity)), so every filtered run prints
872+
what it withheld and glosses the unrated count. Name `UNKNOWN` alongside the
873+
ratings you want to keep them.
874+
- **`--repo` mode has no severities at all**, for the reason in the previous
875+
section, so any `--severity` that omits `UNKNOWN` filters out *everything*.
876+
That does not print as a clean scan:
877+
878+
```console
879+
$ vexscan --repo https://github.com/cwayne18/vexscan --all --severity HIGH,CRITICAL
880+
No findings at these severities.
881+
--severity HIGH,CRITICAL withheld all 1 finding(s): 1 unknown (no rating was published).
882+
This is a filtered view, not a clean result.
883+
```
884+
885+
- **A `--cves` id that matched nothing is never filtered.** Those rows exist so
886+
that an id you named by hand cannot vanish from the report; they carry no
887+
severity, and hiding them would recreate exactly the silence they are there to
888+
prevent.
889+
890+
Exit codes are unchanged: `0` the scan completed, `1` it could not read
891+
something, `2` the command line was wrong. Findings existing — at any severity —
892+
is not a failure, which is what keeps exit `1` worth acting on.
893+
845894
### VEX hubs (`--vexhub`)
846895

847896
Some vendors have already triaged the CVEs in their own images and published the
@@ -931,7 +980,12 @@ The JSON is `schema_version: 2`:
931980
"findings": [ /* flat, sorted — jq '.findings[]' still works */ ],
932981
"ecosystems": [ { "id": "os", "components": 65, "error": "" } ],
933982
"unreadable": { "count": 3, "paths": ["/opt/vendor"] }, // omitted when nothing was skipped
934-
"vex_hubs": [ { "url": "...", "author": "...", "products": 1082, "matched": 3 } ] // only with --vexhub
983+
"vex_hubs": [ { "url": "...", "author": "...", "products": 1082, "matched": 3 } ], // only with --vexhub
984+
"withheld": { // only when --severity hid something; findings[] is already the kept set
985+
"severities": ["CRITICAL", "HIGH"],
986+
"count": 123,
987+
"by_severity": { "UNKNOWN": 36, "MEDIUM": 78, "LOW": 9 }
988+
}
935989
}
936990
```
937991

@@ -995,6 +1049,7 @@ be read, or part of the tree could not be read, `2` the command line was wrong.
9951049
| `--osv-ecosystem` | *(auto)* | Override the OSV ecosystem derived from os-release, e.g. `Debian:12` |
9961050
| `--roots` | | Extra entrypoints for the closures — shared libraries and language imports; repeatable |
9971051
| `--vexhub` | | VEX Repository to check findings against, e.g. `https://github.com/rancher/vexhub` (also a raw base URL or a local directory); repeatable, earliest wins — see [VEX hubs](#vex-hubs---vexhub) |
1052+
| `--severity` | *(all)* | Only report findings at these severities: `CRITICAL`, `HIGH`, `UNKNOWN`, `MEDIUM`, `LOW`, `NONE`; comma-separated or repeatable. `UNKNOWN` must be named to be shown — see [Filtering by severity](#filtering-by-severity---severity) |
9981053
| `--dlopen-policy` | `taint` | `taint` (block conclusions) or `assume-none` |
9991054
| `--dynamic-import-policy` | `taint` | The same knob for a language import graph's computed imports. These are far more common than `dlopen`, so `assume-none` discards much more |
10001055
| `--trust-import-absence` | `false` | Let a missing dynamic import conclude `not_in_execute_path` (weaker than it looks) |

internal/analyze/analyze.go

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,16 @@ type Options struct {
7474
// Ecosystems restricts which plugins run (--ecosystem). Empty runs them
7575
// all. Naming one nothing handles is an error, not an empty result.
7676
Ecosystems []string
77+
// Severities restricts the result to findings carrying these severity
78+
// labels (--severity), already canonicalized through cvss.Parse by the
79+
// caller. Empty keeps everything.
80+
//
81+
// Unlike Ecosystems this changes what is reported rather than what runs:
82+
// every plugin still inventories and every advisory is still resolved,
83+
// because a finding's severity is only knowable once its advisory is in
84+
// hand. What it does buy is that the LLM overlay is never asked about a row
85+
// nobody is going to read.
86+
Severities []string
7787

7888
CVEs []string // optional filter; empty means "every advisory that applies"
7989
Version string // optional override of the detected module version (image mode)
@@ -166,6 +176,11 @@ type Result struct {
166176
// not be read. It is not part of Failed(): see vexOverlay for why a hub
167177
// failure is not the same kind of incompleteness as an ecosystem failure.
168178
VEXHubs []ecosystem.VEXHubResult `json:"vex_hubs,omitempty"`
179+
180+
// Withheld is what --severity removed from Findings, and is nil when the
181+
// flag was not used or hid nothing. See severityFilter: a filtered result
182+
// and a clean one are indistinguishable without it.
183+
Withheld *Withheld `json:"withheld,omitempty"`
169184
}
170185

171186
// Failed reports whether the findings are an incomplete account of the target
@@ -486,6 +501,11 @@ func runTree(ctx context.Context, opts Options) (*Result, error) {
486501
}
487502

488503
severityOverlay(result.Findings, run.resolver.severities())
504+
// Filtering here, rather than in the renderer, is what keeps every count
505+
// downstream honest: the LLM is never billed for a row nobody will read,
506+
// and a hub's Matched is statements about findings that are actually in the
507+
// report rather than ones that had already been dropped.
508+
result.Findings, result.Withheld = severityFilter(result.Findings, opts.Severities)
489509
// The image is only a product for a scan that was given one: --rootfs
490510
// analyzes a tree whose provenance nobody recorded, and inventing a purl
491511
// from a directory name would look up an artifact that does not exist.
@@ -628,6 +648,12 @@ func runRepo(ctx context.Context, opts Options) (*Result, error) {
628648
result.Findings = append(result.Findings, unmapped(opts.CVEs, result.Findings)...)
629649

630650
severityOverlay(result.Findings, run.resolver.severities())
651+
// See runTree for why the filter runs before the overlays rather than in
652+
// the renderer. Repo mode is the path where it bites hardest: govulncheck's
653+
// OpenVEX carries no severity, so every Go finding here is UNKNOWN and a
654+
// --severity that does not name UNKNOWN empties the report completely. The
655+
// renderer has to say so, which is what Withheld is for.
656+
result.Findings, result.Withheld = severityFilter(result.Findings, opts.Severities)
631657
// No productOverlay here: repo mode has no image, and the only artifact a
632658
// checkout is is its own module, which the Go plugin already recorded.
633659
result.VEXHubs = vexOverlay(ctx, opts.VEXHubs, result.Findings, run.resolver.aliases(), logf)

internal/analyze/severity.go

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
package analyze
2+
3+
import (
4+
"github.com/cwayne18/vexscan/internal/cvss"
5+
"github.com/cwayne18/vexscan/internal/ecosystem"
6+
)
7+
8+
// Withheld records what --severity removed from the result.
9+
//
10+
// It exists because a filtered report and a clean one look identical, and that
11+
// is the one confusion this tool must never cause. Every renderer prints this
12+
// before the findings, so a short list is always accompanied by the reason it
13+
// is short.
14+
//
15+
// It is deliberately not part of Failed(). The scan completed and read
16+
// everything it meant to; the reader asked for a subset of what it found. That
17+
// is the opposite of an ecosystem that could not be inventoried, where the tool
18+
// does not know what it missed.
19+
type Withheld struct {
20+
// Severities is what --severity asked to keep, so the banner can quote the
21+
// flag back rather than making the reader remember what they typed.
22+
Severities []string `json:"severities"`
23+
Count int `json:"count"`
24+
// BySeverity is what was dropped, keyed by label. UNKNOWN in here is the
25+
// entry that matters: those findings are unrated, not unimportant.
26+
BySeverity map[string]int `json:"by_severity"`
27+
}
28+
29+
// severityFilter keeps only the findings whose severity was asked for, and
30+
// reports what it dropped.
31+
//
32+
// An empty keep-list is a no-op returning a nil Withheld, which is how an
33+
// unfiltered run produces neither a JSON field nor a banner.
34+
//
35+
// Matching is on cvss.Display, so a finding no advisory resolved for and one
36+
// whose advisory published no rating are both UNKNOWN here -- the same fact
37+
// they already are to a reader. UNKNOWN is therefore a severity you can ask
38+
// for, and one you have to ask for: --severity CRITICAL,HIGH drops it. That is
39+
// Trivy's behavior and it is what a reader of the flag expects, but it is worth
40+
// being clear-eyed that on debian:12 it hides 36 findings whose only crime is
41+
// that their record is CVSS v4-only, and on --repo it hides every Go finding
42+
// there is. The banner exists because of that, not in spite of it.
43+
func severityFilter(findings []Finding, keep []string) ([]Finding, *Withheld) {
44+
if len(keep) == 0 {
45+
return findings, nil
46+
}
47+
wanted := make(map[string]bool, len(keep))
48+
for _, s := range keep {
49+
wanted[cvss.Display(s)] = true
50+
}
51+
52+
kept := make([]Finding, 0, len(findings))
53+
w := &Withheld{Severities: keep, BySeverity: map[string]int{}}
54+
for _, f := range findings {
55+
if wanted[cvss.Display(f.Severity)] || alwaysReport(f) {
56+
kept = append(kept, f)
57+
continue
58+
}
59+
w.Count++
60+
w.BySeverity[cvss.Display(f.Severity)]++
61+
}
62+
if w.Count == 0 {
63+
// Nothing was hidden, so there is nothing to warn about. A banner
64+
// saying "withheld 0 findings" is noise that trains a reader to skip
65+
// the line that will one day say something.
66+
return kept, nil
67+
}
68+
return kept, w
69+
}
70+
71+
// alwaysReport is the one exemption from the filter: a finding that exists to
72+
// account for an id the user named by hand.
73+
//
74+
// unmapped emits these so that a --cves id which matched no component anywhere
75+
// still appears in the output, on the grounds that "a missing id reads as a
76+
// clean one". They carry no severity, so a severity filter would delete every
77+
// one of them and recreate exactly the silence unmapped was written to prevent
78+
// -- and it would do it to the ids the reader was most explicitly asking about.
79+
func alwaysReport(f Finding) bool {
80+
return f.Reason == "no_component_matched" && f.Status == ecosystem.StatusUndetermined
81+
}

internal/analyze/severity_test.go

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,3 +127,149 @@ func TestSeverityOverlayLeavesUnresolvableFindingsAlone(t *testing.T) {
127127
t.Errorf("unresolved finding was labelled: %+v", findings[0])
128128
}
129129
}
130+
131+
// rated is a linked finding at one severity, which is all severityFilter looks
132+
// at. An empty label is the real and common case: no advisory was resolved.
133+
func rated(id, label string) Finding {
134+
return Finding{
135+
Ecosystem: "os", ID: id, CVE: id,
136+
Severity: label, Status: StatusLinked,
137+
}
138+
}
139+
140+
func labels(findings []Finding) []string {
141+
out := make([]string, 0, len(findings))
142+
for _, f := range findings {
143+
out = append(out, f.ID)
144+
}
145+
return out
146+
}
147+
148+
func TestNoSeverityFilterIsANoOp(t *testing.T) {
149+
in := []Finding{rated("a", cvss.Critical), rated("b", cvss.Low)}
150+
got, w := severityFilter(in, nil)
151+
if len(got) != 2 {
152+
t.Errorf("kept %v, want both", labels(got))
153+
}
154+
if w != nil {
155+
t.Errorf("Withheld = %+v, want nil so no banner prints", w)
156+
}
157+
}
158+
159+
func TestSeverityFilterKeepsOnlyWhatWasNamed(t *testing.T) {
160+
in := []Finding{
161+
rated("crit", cvss.Critical),
162+
rated("high", cvss.High),
163+
rated("med", cvss.Medium),
164+
rated("low", cvss.Low),
165+
}
166+
got, w := severityFilter(in, []string{cvss.Critical, cvss.High})
167+
if len(got) != 2 || got[0].ID != "crit" || got[1].ID != "high" {
168+
t.Errorf("kept %v, want [crit high]", labels(got))
169+
}
170+
if w == nil || w.Count != 2 {
171+
t.Fatalf("Withheld = %+v, want 2 dropped", w)
172+
}
173+
if w.BySeverity[cvss.Medium] != 1 || w.BySeverity[cvss.Low] != 1 {
174+
t.Errorf("BySeverity = %v, want one medium and one low", w.BySeverity)
175+
}
176+
}
177+
178+
// The decision that most surprises a Trivy user, pinned: an unrated finding is
179+
// dropped by --severity CRITICAL,HIGH, and counted as UNKNOWN so the banner can
180+
// say the drop happened.
181+
func TestAnUnratedFindingIsWithheldAsUnknown(t *testing.T) {
182+
in := []Finding{rated("crit", cvss.Critical), rated("unrated", "")}
183+
got, w := severityFilter(in, []string{cvss.Critical, cvss.High})
184+
if len(got) != 1 || got[0].ID != "crit" {
185+
t.Errorf("kept %v, want [crit]", labels(got))
186+
}
187+
if w == nil || w.BySeverity[cvss.Unknown] != 1 {
188+
t.Fatalf("Withheld = %+v, want the unrated finding counted as UNKNOWN", w)
189+
}
190+
}
191+
192+
// An empty Severity and an explicit UNKNOWN are the same fact to a reader, so
193+
// naming UNKNOWN has to reach both.
194+
func TestNamingUnknownKeepsBothSpellingsOfUnrated(t *testing.T) {
195+
in := []Finding{
196+
rated("empty", ""),
197+
rated("explicit", cvss.Unknown),
198+
rated("high", cvss.High),
199+
}
200+
got, w := severityFilter(in, []string{cvss.Unknown})
201+
if len(got) != 2 || got[0].ID != "empty" || got[1].ID != "explicit" {
202+
t.Errorf("kept %v, want both unrated spellings", labels(got))
203+
}
204+
if w == nil || w.Count != 1 || w.BySeverity[cvss.High] != 1 {
205+
t.Errorf("Withheld = %+v, want only the high one dropped", w)
206+
}
207+
}
208+
209+
// The exemption. unmapped emits this row so an id the user typed cannot vanish;
210+
// a severity filter deleting it would recreate exactly the silence that row
211+
// exists to prevent, and would do it to the id they asked about most directly.
212+
func TestARequestedIdThatMatchedNothingSurvivesTheFilter(t *testing.T) {
213+
in := append(unmapped([]string{"CVE-2024-9999"}, nil), rated("med", cvss.Medium))
214+
got, w := severityFilter(in, []string{cvss.Critical})
215+
if len(got) != 1 || got[0].CVE != "CVE-2024-9999" {
216+
t.Fatalf("kept %v, want the unmatched id to survive", labels(got))
217+
}
218+
if got[0].Reason != "no_component_matched" {
219+
t.Errorf("Reason = %q, want it unchanged", got[0].Reason)
220+
}
221+
// It survived, so it is not counted as withheld -- the banner would be
222+
// claiming to have hidden a row that is printed right below it.
223+
if w == nil || w.Count != 1 || w.BySeverity[cvss.Medium] != 1 {
224+
t.Errorf("Withheld = %+v, want only the medium finding counted", w)
225+
}
226+
}
227+
228+
// An undetermined finding with no severity is not exempt: only the
229+
// no_component_matched receipt is.
230+
func TestTheExemptionIsNarrow(t *testing.T) {
231+
f := Finding{ID: "x", CVE: "x", Status: StatusUndetermined, Reason: "dlopen_reachable"}
232+
got, w := severityFilter([]Finding{f}, []string{cvss.Critical})
233+
if len(got) != 0 {
234+
t.Errorf("kept %v, want an ordinary undetermined finding to be filtered", labels(got))
235+
}
236+
if w == nil || w.Count != 1 {
237+
t.Errorf("Withheld = %+v, want it counted", w)
238+
}
239+
}
240+
241+
// A filter that hid nothing produces no banner, so the line that will one day
242+
// matter is not one a reader has been trained to skip.
243+
func TestAFilterThatHidNothingReportsNothing(t *testing.T) {
244+
in := []Finding{rated("crit", cvss.Critical)}
245+
got, w := severityFilter(in, []string{cvss.Critical, cvss.High})
246+
if len(got) != 1 {
247+
t.Errorf("kept %v, want the critical finding", labels(got))
248+
}
249+
if w != nil {
250+
t.Errorf("Withheld = %+v, want nil when nothing was dropped", w)
251+
}
252+
}
253+
254+
func TestWithheldQuotesTheFlagBack(t *testing.T) {
255+
_, w := severityFilter([]Finding{rated("low", cvss.Low)}, []string{cvss.Critical, cvss.High})
256+
if w == nil {
257+
t.Fatal("want a Withheld")
258+
}
259+
if len(w.Severities) != 2 || w.Severities[0] != cvss.Critical || w.Severities[1] != cvss.High {
260+
t.Errorf("Severities = %v, want what --severity asked for", w.Severities)
261+
}
262+
}
263+
264+
// Filtering everything is legal and is the --repo case: every Go finding there
265+
// is UNKNOWN. What must not happen is it going unrecorded.
266+
func TestFilteringEverythingIsStillRecorded(t *testing.T) {
267+
in := []Finding{rated("a", ""), rated("b", "")}
268+
got, w := severityFilter(in, []string{cvss.High, cvss.Critical})
269+
if len(got) != 0 {
270+
t.Errorf("kept %v, want nothing", labels(got))
271+
}
272+
if w == nil || w.Count != 2 || w.BySeverity[cvss.Unknown] != 2 {
273+
t.Fatalf("Withheld = %+v, want both counted as unrated", w)
274+
}
275+
}

0 commit comments

Comments
 (0)