Skip to content

Commit 654bd5c

Browse files
authored
Add Support for Security Gating using Filters (#9)
* Add support for ignorable directories * Update DRY snapshot * Support log redirection to file * Add support to redirect log to stdout * Add summary reporter * Refactor scan method * Refactor to introduce Finish method in analyzer * Refactor to conslidated reporting for CEL filter * Show unique CEL filter results * Add support for filter fail option * Fix README
1 parent 52c72ed commit 654bd5c

18 files changed

Lines changed: 567 additions & 76 deletions

File tree

README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -53,23 +53,23 @@ vet scan --lockfiles /path/to/package-lock.json
5353

5454
> Use `vet scan parsers` to list supported package manifest parsers
5555
56-
The default scan uses an opinionated [Console Reporter](#) which presents
57-
a summary of findings per package manifest. Thats NOT about it. Read more for
56+
The default scan uses an opinionated [Summary Reporter](#) which presents
57+
a consolidated summary of findings. Thats NOT about it. Read more for
5858
expression based filtering and policy evaluation.
5959

6060
## Filtering
6161

6262
Find dependencies that seems not very popular
6363

6464
```bash
65-
vet scan --lockfiles /path/to/pom.xml --report-console=false \
65+
vet scan --lockfiles /path/to/pom.xml --report-summary=false \
6666
--filter='projects.exists(x, x.stars < 10)'
6767
```
6868

6969
Find dependencies with a critical vulnerability
7070

7171
```bash
72-
vet scan --lockfiles /path/to/pom.xml --report-console=false \
72+
vet scan --lockfiles /path/to/pom.xml --report-summary=false \
7373
--filter='vulns.critical.exists_one(x, true)'
7474
```
7575

docs/filtering.md

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,14 @@ Filter command helps solve the problem of visibility for OSS dependencies in an
44
application. To support various requirements, we adopt a generic [expressions
55
language](https://github.com/google/cel-spec) for flexible filtering.
66

7+
Example:
8+
9+
```bash
10+
vet scan -D /path/to/repo \
11+
--report-summary=false \
12+
--filter 'licenses.exists(p, p == "MIT")'
13+
```
14+
715
## Input
816

917
Filter expressions work on packages (aka. dependencies) and evaluates to
@@ -43,7 +51,7 @@ a list of checks available from OpenSSF Scorecards project.
4351

4452
Scanning a package manifest is a resource intensive process as it involves
4553
enriching package metadata by queryin [Insights API](https://safedep.io/docs/concepts/raya-data-platform-overview).
46-
However, for filtering and reporting may be done multiple times on the same
54+
However, filtering and reporting may be done multiple times on the same
4755
manifest. To speed up the process, we can dump the enriched data as JSON and
4856
load the same for filtering and reporting.
4957

@@ -57,10 +65,31 @@ vet scan -D /path/to/repository --json-dump-dir /tmp/dump-many
5765
Load the enriched metadata for filtering and reporting
5866

5967
```bash
60-
vet query --from /tmp/dump --report-console
68+
vet query --from /tmp/dump --report-summary
6169
vet query --from /tmp/dump --filter 'scorecard.score.Maintained == 0'
6270
```
6371

72+
## Gating with Filters
73+
74+
A simple security gate (in CI) can be achieved using the filters. The
75+
`--filter-fail` argument tells the `Filter Analyzer` module to fail the command
76+
if any package matches the given filter.
77+
78+
Example:
79+
80+
```bash
81+
vet query --from /path/to/json-dump \
82+
--filter 'scorecard.scores.Maintained == 0' \
83+
--filter-fail
84+
```
85+
86+
Subsequently, the command fails with `-1` exit code in case of match
87+
88+
```bash
89+
➜ vet git:(develop) ✗ echo $?
90+
255
91+
```
92+
6493
## FAQ
6594

6695
### How does the filter input JSON look like?

go.mod

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,26 +4,27 @@ go 1.18
44

55
require (
66
github.com/deepmap/oapi-codegen v1.12.4
7+
github.com/golang/protobuf v1.5.2
78
github.com/google/cel-go v0.13.0
89
github.com/google/osv-scanner v1.1.0
910
github.com/jedib0t/go-pretty/v6 v6.4.4
10-
github.com/safedep/dry v0.0.0-20230202121135-2225c66946de
11+
github.com/safedep/dry v0.0.0-20230203134955-367834d99b1c
1112
github.com/sirupsen/logrus v1.9.0
1213
github.com/spf13/cobra v1.6.1
1314
github.com/stretchr/testify v1.8.1
1415
golang.org/x/term v0.4.0
16+
google.golang.org/protobuf v1.28.1
1517
gopkg.in/yaml.v2 v2.4.0
1618
)
1719

1820
require (
1921
github.com/BurntSushi/toml v1.2.1 // indirect
22+
github.com/Masterminds/semver/v3 v3.2.0 // indirect
2023
github.com/antlr/antlr4/runtime/Go/antlr v1.4.10 // indirect
2124
github.com/apapsch/go-jsonmerge/v2 v2.0.0 // indirect
2225
github.com/davecgh/go-spew v1.1.1 // indirect
23-
github.com/golang/protobuf v1.5.2 // indirect
2426
github.com/google/uuid v1.3.0 // indirect
2527
github.com/inconshreveable/mousetrap v1.0.1 // indirect
26-
github.com/masterminds/semver v1.5.0 // indirect
2728
github.com/mattn/go-runewidth v0.0.13 // indirect
2829
github.com/mitchellh/mapstructure v1.5.0 // indirect
2930
github.com/oklog/ulid/v2 v2.1.0 // indirect
@@ -35,6 +36,5 @@ require (
3536
golang.org/x/sys v0.4.0 // indirect
3637
golang.org/x/text v0.5.0 // indirect
3738
google.golang.org/genproto v0.0.0-20221027153422-115e99e71e1c // indirect
38-
google.golang.org/protobuf v1.28.1 // indirect
3939
gopkg.in/yaml.v3 v3.0.1 // indirect
4040
)

go.sum

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
github.com/BurntSushi/toml v1.2.1 h1:9F2/+DoOYIOksmaJFPw1tGFy1eDnIJXg+UHjuD8lTak=
22
github.com/BurntSushi/toml v1.2.1/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ=
3-
github.com/Masterminds/semver v1.5.0 h1:H65muMkzWKEuNDnfl9d70GUjFniHKHRbFPGBuZ3QEww=
3+
github.com/Masterminds/semver/v3 v3.2.0 h1:3MEsd0SM6jqZojhjLWWeBY+Kcjy9i6MQAeY7YgDP83g=
4+
github.com/Masterminds/semver/v3 v3.2.0/go.mod h1:qvl/7zhW3nngYb5+80sSMF+FG2BjYrf8m9wsX0PNOMQ=
45
github.com/RaveNoX/go-jsoncommentstrip v1.0.0/go.mod h1:78ihd09MekBnJnxpICcwzCMzGrKSKYe4AqU6PDYYpjk=
56
github.com/antlr/antlr4/runtime/Go/antlr v1.4.10 h1:yL7+Jz0jTC6yykIK/Wh74gnTJnrGr5AyrNMXuA0gves=
67
github.com/antlr/antlr4/runtime/Go/antlr v1.4.10/go.mod h1:F7bn7fEU90QkQ3tnmaTx3LTKLEDqnwWODIYppRQ5hnY=
@@ -29,8 +30,6 @@ github.com/inconshreveable/mousetrap v1.0.1/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLf
2930
github.com/jedib0t/go-pretty/v6 v6.4.4 h1:N+gz6UngBPF4M288kiMURPHELDMIhF/Em35aYuKrsSc=
3031
github.com/jedib0t/go-pretty/v6 v6.4.4/go.mod h1:MgmISkTWDSFu0xOqiZ0mKNntMQ2mDgOcwOkwBEkMDJI=
3132
github.com/juju/gnuflag v0.0.0-20171113085948-2ce1bb71843d/go.mod h1:2PavIy+JPciBPrBUjwbNvtwB6RQlve+hkpll6QSNmOE=
32-
github.com/masterminds/semver v1.5.0 h1:hTxJTTY7tjvnWMrl08O6u3G6BLlKVwxSz01lVac9P8U=
33-
github.com/masterminds/semver v1.5.0/go.mod h1:s7KNT9fnd7edGzwwP7RBX4H0v/CYd5qdOLfkL1V75yg=
3433
github.com/mattn/go-runewidth v0.0.13 h1:lTGmDsbAYt5DmK6OnoV7EuIF1wEIFAcxld6ypU4OSgU=
3534
github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
3635
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
@@ -44,8 +43,8 @@ github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZN
4443
github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
4544
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
4645
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
47-
github.com/safedep/dry v0.0.0-20230202121135-2225c66946de h1:07LwA5P5bxaVK8SXtoBfP8qZGcWAtqiFbq54UZnMcyg=
48-
github.com/safedep/dry v0.0.0-20230202121135-2225c66946de/go.mod h1:H111d9khzpHFEqKXb9lgAZ1W5eeK7yEN7ToWu0ak+JI=
46+
github.com/safedep/dry v0.0.0-20230203134955-367834d99b1c h1:zbhTBT463mwcIuCq89GT8pFTU8UtGalBWFaa/wsgVXA=
47+
github.com/safedep/dry v0.0.0-20230203134955-367834d99b1c/go.mod h1:yZ8R6kv4pR0yertVoxgBmnN4bvHT8TLubE7aahpWDDk=
4948
github.com/sirupsen/logrus v1.9.0 h1:trlNQbNUG3OdDrDil03MCb1H2o9nJ1x4/5LYw7byDE0=
5049
github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
5150
github.com/spf13/cobra v1.6.1 h1:o94oiPyS4KD1mPy2fmcYYHHfCxLqYjJOhGsCHFZtEzA=

main.go

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,16 +2,19 @@ package main
22

33
import (
44
"fmt"
5+
"io/ioutil"
56
"os"
67
"strconv"
78

9+
"github.com/safedep/dry/utils"
810
"github.com/safedep/vet/pkg/common/logger"
911
"github.com/spf13/cobra"
1012
)
1113

1214
var (
1315
verbose bool
1416
debug bool
17+
logFile string
1518
)
1619

1720
var banner string = `
@@ -45,6 +48,7 @@ func main() {
4548

4649
cmd.PersistentFlags().BoolVarP(&verbose, "verbose", "v", false, "Show verbose logs")
4750
cmd.PersistentFlags().BoolVarP(&debug, "debug", "d", false, "Show debug logs")
51+
cmd.PersistentFlags().StringVarP(&logFile, "log", "l", "", "Write command logs to file")
4852

4953
cmd.AddCommand(newAuthCommand())
5054
cmd.AddCommand(newScanCommand())
@@ -67,3 +71,23 @@ func printBanner() {
6771
fmt.Print(banner)
6872
}
6973
}
74+
75+
// Redirect to file or discard log if empty
76+
func redirectLogToFile(path string) {
77+
if !utils.IsEmptyString(path) {
78+
if path == "-" {
79+
logger.MigrateTo(os.Stdout)
80+
} else {
81+
logger.LogToFile(path)
82+
}
83+
} else {
84+
logger.MigrateTo(ioutil.Discard)
85+
}
86+
}
87+
88+
func failOnError(stage string, err error) {
89+
if err != nil {
90+
logger.Errorf("%s failed due to error %v", stage, err)
91+
os.Exit(-1)
92+
}
93+
}

pkg/analyzer/analyzer.go

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ type AnalyzerEventType string
66

77
const (
88
ET_FilterExpressionMatched = AnalyzerEventType("ev_pkg_filter_match")
9+
ET_AnalyzerFailOnError = AnalyzerEventType("ev_fail_on_error")
910
)
1011

1112
type AnalyzerEvent struct {
@@ -15,6 +16,10 @@ type AnalyzerEvent struct {
1516
// Type of the event
1617
Type AnalyzerEventType
1718

19+
// Message / Error
20+
Message interface{}
21+
Err error
22+
1823
// Entities on which event was generated
1924
Manifest *models.PackageManifest
2025
Package *models.Package
@@ -29,4 +34,6 @@ type Analyzer interface {
2934

3035
Analyze(manifest *models.PackageManifest,
3136
handler AnalyzerEventHandler) error
37+
38+
Finish() error
3239
}

pkg/analyzer/cel_filter.go

Lines changed: 66 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -28,10 +28,20 @@ const (
2828
)
2929

3030
type celFilterAnalyzer struct {
31-
program cel.Program
31+
program cel.Program
32+
failOnMatch bool
33+
34+
packages map[string]*models.Package
35+
36+
stat struct {
37+
manifests int
38+
packages int
39+
matched int
40+
err int
41+
}
3242
}
3343

34-
func NewCelFilterAnalyzer(filter string) (Analyzer, error) {
44+
func NewCelFilterAnalyzer(filter string, failOnMatch bool) (Analyzer, error) {
3545
env, err := cel.NewEnv(
3646
cel.Variable(filterInputVarPkg, cel.DynType),
3747
cel.Variable(filterInputVarVulns, cel.DynType),
@@ -55,7 +65,10 @@ func NewCelFilterAnalyzer(filter string) (Analyzer, error) {
5565
return nil, err
5666
}
5767

58-
return &celFilterAnalyzer{program: prog}, nil
68+
return &celFilterAnalyzer{program: prog,
69+
failOnMatch: failOnMatch,
70+
packages: make(map[string]*models.Package),
71+
}, nil
5972
}
6073

6174
func (f *celFilterAnalyzer) Name() string {
@@ -65,32 +78,22 @@ func (f *celFilterAnalyzer) Name() string {
6578
func (f *celFilterAnalyzer) Analyze(manifest *models.PackageManifest,
6679
handler AnalyzerEventHandler) error {
6780

68-
tbl := table.NewWriter()
69-
tbl.SetStyle(table.StyleLight)
70-
tbl.SetOutputMirror(os.Stdout)
71-
tbl.AppendHeader(table.Row{"Ecosystem", "Package", "Version",
72-
"Latest", "Source"})
73-
74-
filterStat := struct {
75-
total int
76-
matched int
77-
err int
78-
}{}
79-
8081
logger.Infof("CEL filtering manifest: %s", manifest.Path)
82+
f.stat.manifests += 1
83+
8184
for _, pkg := range manifest.Packages {
82-
filterStat.total += 1
85+
f.stat.packages += 1
8386

8487
filterInput, err := f.buildFilterInput(pkg)
8588
if err != nil {
86-
filterStat.err += 1
89+
f.stat.err += 1
8790
logger.Errorf("Failed to convert package to filter input: %v", err)
8891
continue
8992
}
9093

9194
serializedInput, err := f.serializeFilterInput(filterInput)
9295
if err != nil {
93-
filterStat.err += 1
96+
f.stat.err += 1
9497
logger.Errorf("Failed to serialize filter input: %v", err)
9598
continue
9699
}
@@ -105,7 +108,7 @@ func (f *celFilterAnalyzer) Analyze(manifest *models.PackageManifest,
105108
})
106109

107110
if err != nil {
108-
filterStat.err += 1
111+
f.stat.err += 1
109112
logger.Errorf("Failed to evaluate CEL for %s:%v : %v",
110113
pkg.PackageDetails.Name,
111114
pkg.PackageDetails.Version, err)
@@ -114,24 +117,59 @@ func (f *celFilterAnalyzer) Analyze(manifest *models.PackageManifest,
114117

115118
if (reflect.TypeOf(out).Kind() == reflect.Bool) &&
116119
(reflect.ValueOf(out).Bool()) {
117-
filterStat.matched += 1
118-
tbl.AppendRow(table.Row{pkg.PackageDetails.Ecosystem,
119-
pkg.PackageDetails.Name,
120-
pkg.PackageDetails.Version,
121-
f.pkgLatestVersion(pkg),
122-
f.pkgSource(pkg),
123-
})
120+
121+
// Avoid duplicates added to the table
122+
if _, ok := f.packages[pkg.Id()]; ok {
123+
continue
124+
}
125+
126+
f.stat.matched += 1
127+
f.packages[pkg.Id()] = pkg
124128
}
125129
}
126130

131+
return f.notifyCaller(manifest, handler)
132+
}
133+
134+
func (f *celFilterAnalyzer) Finish() error {
135+
tbl := table.NewWriter()
136+
tbl.SetStyle(table.StyleLight)
137+
tbl.SetOutputMirror(os.Stdout)
138+
tbl.AppendHeader(table.Row{"Ecosystem", "Package", "Version",
139+
"Source"})
140+
141+
for _, pkg := range f.packages {
142+
tbl.AppendRow(table.Row{pkg.PackageDetails.Ecosystem,
143+
pkg.PackageDetails.Name,
144+
pkg.PackageDetails.Version,
145+
f.pkgSource(pkg),
146+
})
147+
}
148+
127149
fmt.Printf("%s\n", text.Bold.Sprint("Filter evaluated with ",
128-
filterStat.matched, " out of ", filterStat.total, " matched and ",
129-
filterStat.err, " error(s)"))
150+
f.stat.matched, " out of ", f.stat.packages, " uniquely matched and ",
151+
f.stat.err, " error(s) ", "across ", f.stat.manifests,
152+
" manifest(s)"))
130153

131154
tbl.Render()
132155
return nil
133156
}
134157

158+
func (f *celFilterAnalyzer) notifyCaller(manifest *models.PackageManifest,
159+
handler AnalyzerEventHandler) error {
160+
if f.failOnMatch && (f.stat.matched > 0) {
161+
handler(&AnalyzerEvent{
162+
Source: f.Name(),
163+
Type: ET_AnalyzerFailOnError,
164+
Manifest: manifest,
165+
Err: fmt.Errorf("failed due to filter match on %s",
166+
manifest.Path),
167+
})
168+
}
169+
170+
return nil
171+
}
172+
135173
// TODO: Fix this JSON round-trip problem by directly configuring CEL env to
136174
// work with Protobuf messages
137175
func (f *celFilterAnalyzer) serializeFilterInput(fi *filterinput.FilterInput) (map[string]interface{}, error) {

pkg/analyzer/event.go

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
package analyzer
2+
3+
func (ev *AnalyzerEvent) IsFailOnError() bool {
4+
return ev.Type == ET_AnalyzerFailOnError
5+
}

pkg/analyzer/json_dump.go

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,3 +56,7 @@ func (j *jsonDumperAnalyzer) Analyze(manifest *models.PackageManifest,
5656

5757
return ioutil.WriteFile(path, data, 0600)
5858
}
59+
60+
func (j *jsonDumperAnalyzer) Finish() error {
61+
return nil
62+
}

0 commit comments

Comments
 (0)