Skip to content

Commit b10cce3

Browse files
authored
Merge pull request #133 from serranos/add-bundle-audit-tests
Add unit tests for bundle-audit
2 parents 53bbad7 + 2c3781a commit b10cce3

File tree

14 files changed

+380
-0
lines changed

14 files changed

+380
-0
lines changed

spec/tasks/bundle-audit/README.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
## bundler-audit spec tests
2+
3+
### Overview of bundler-audit
4+
5+
[bundler-audit](https://github.com/rubysec/bundler-audit)
6+
7+
Scans a project's vulnerable versions of gems in the `Gemfile.lock` file and checks for gem sources without TLS.
8+
9+
The names/versions are compared against the [ruby-advisory-db ](https://github.com/rubysec/ruby-advisory-db).
10+
11+
To install bundler-audit:
12+
```
13+
gem install bundler-audit
14+
```
15+
16+
The simplest way to run it from the command line is to `cd` to the folder with the `Gemfile.lock` file and call:
17+
```
18+
bundle-audit check
19+
```
20+
21+
In Glue, `bundler-audit` is called with the following argument:
22+
```
23+
bundle-audit check
24+
```
25+
26+
Some other command line options when running bundler-audit locally:
27+
* `--update` - Updates the ruby-advisory-db;
28+
* `--ignore [ADVISORY-ID]` - Ignores specific advisories.
29+
30+
See [bundler-audit documentation](https://www.rubydoc.info/gems/bundler-audit/frames) for more info.
31+
32+
### The spec tests
33+
34+
The specs do not call the bundler-audit tool because this would be too slow (~1 sec per spec test).
35+
Instead the specs rely on stubbing Glue's `runsystem` method (which calls CLI commands).
36+
37+
In the specs, the return value of `runsystem` is always a canned response.
38+
Either it will be a generic, minimal response, or it will be a snapshot of an actual bundler-audit report.
39+
40+
The actual reports were generated via the script `generate_reports.sh`.
41+
The targets of the spec tests were set up in a minimal way to produce non-trivial output.
42+
This required a `Gemfile.lock` file per target.
Lines changed: 209 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
require 'spec_helper'
2+
require 'rspec/collection_matchers'
3+
require 'glue'
4+
require 'glue/event'
5+
require 'glue/tracker'
6+
require 'glue/tasks'
7+
require 'glue/tasks/bundle-audit'
8+
9+
describe Glue::BundleAudit do
10+
11+
BUNDLEAUDIT_TARGETS_PATH = 'spec/tasks/bundle-audit/targets'
12+
13+
def get_bundleaudit(target = 'nil_target')
14+
trigger = Glue::Event.new(target)
15+
trigger.path = File.join(BUNDLEAUDIT_TARGETS_PATH, target)
16+
tracker = Glue::Tracker.new({})
17+
Glue::BundleAudit.new(trigger, tracker)
18+
end
19+
20+
def get_raw_report(target, subtarget = nil)
21+
path = File.join(get_target_path(target, subtarget), "report.txt")
22+
File.read(path).chomp
23+
end
24+
25+
def get_target_path(target, subtarget = nil)
26+
if subtarget.nil?
27+
File.join(BUNDLEAUDIT_TARGETS_PATH, target)
28+
else
29+
File.join(BUNDLEAUDIT_TARGETS_PATH, target, subtarget)
30+
end
31+
end
32+
33+
def cli_args(target, subtarget = nil)
34+
[true, "bundle-audit", "check", { chdir: get_target_path(target, subtarget) }]
35+
end
36+
37+
describe "#initialize" do
38+
let(:task) { @task }
39+
before(:all) { @task = get_bundleaudit }
40+
41+
it "sets the correct 'name'" do
42+
expect(task.name).to eq('BundleAudit')
43+
end
44+
45+
it "sets the correct 'stage'" do
46+
expect(task.stage).to eq(:code)
47+
end
48+
49+
it "sets the correct 'labels'" do
50+
expect(task.labels).to eq(%w[code ruby].to_set)
51+
end
52+
end
53+
54+
describe '#supported?' do
55+
subject(:task) { get_bundleaudit }
56+
57+
context "when 'runsystem' cannot run the task" do
58+
before do
59+
allow(Glue).to receive(:notify) # suppress the output
60+
cmd_args = [false, "bundle-audit", "update"]
61+
cmd_str = 'command not found'
62+
allow(task).to receive(:runsystem).with(*cmd_args).and_return(cmd_str)
63+
end
64+
65+
it { is_expected.not_to be_supported }
66+
67+
it 'issues a notification' do
68+
expect(Glue).to receive(:notify)
69+
task.supported?
70+
end
71+
end
72+
73+
context "when 'runsystem' returns an updating message" do
74+
before do
75+
cmd_args = [false, "bundle-audit", "update"]
76+
cmd_str = 'Updating ruby-advisory-db ...'
77+
allow(task).to receive(:runsystem).with(*cmd_args).and_return(cmd_str)
78+
end
79+
80+
it { is_expected.to be_supported }
81+
end
82+
end
83+
84+
describe "#run" do
85+
let(:task) { get_bundleaudit target }
86+
let(:minimal_response) { "No vulnerabilities found" }
87+
88+
before do
89+
allow(Glue).to receive(:notify) # suppress the output
90+
allow(Glue).to receive(:warn) # suppress the output
91+
92+
# This acts as a guard against actually calling bundle-audit from the CLI.
93+
# (All specs should use canned responses instead.)
94+
allow(task).to receive(:runsystem) do
95+
puts "Warning from rspec -- make sure you're not attempting to call the actual bundle-audit CLI"
96+
puts "within an 'it' block with description '#{self.class.description}'"
97+
minimal_response
98+
end
99+
end
100+
101+
context "with no Gemfile.lock file in root, and no sub-dirs" do
102+
let(:target) { 'no_findings_no_gemfile_lock' }
103+
104+
it "does not call bundle-audit on the target" do
105+
expect(task).not_to receive(:runsystem).with(*cli_args(target))
106+
task.run
107+
end
108+
end
109+
110+
context 'assuming valid (but minimal) reports' do
111+
context 'with one Gemfile.lock in the root dir' do
112+
let(:target) { 'finding_1' }
113+
114+
before do
115+
allow(task).to receive(:runsystem).with(*cli_args(target))
116+
end
117+
118+
it 'passes the task name to Glue.notify' do
119+
expect(Glue).to receive(:notify).with(/^BundleAudit/)
120+
task.run
121+
end
122+
123+
it "calls the 'bundle-audit' cli once, from the root dir" do
124+
expect(task).to receive(:runsystem).with(*cli_args(target))
125+
task.run
126+
end
127+
128+
end
129+
130+
end
131+
end
132+
133+
describe "#analyze" do
134+
let(:task) { get_bundleaudit target }
135+
let(:minimal_response) { "No vulnerabilities found" }
136+
137+
before do
138+
allow(Glue).to receive(:notify) # suppress the output
139+
allow(Glue).to receive(:warn) # suppress the output
140+
141+
# This acts as a guard against actually calling bundle-audit from the CLI.
142+
# (All specs should use canned responses instead.)
143+
allow(task).to receive(:runsystem) do
144+
puts "Warning from rspec -- make sure you're not attempting to call the actual bundle-audit API"
145+
puts "within an 'it' block with description '#{self.class.description}'"
146+
minimal_response
147+
end
148+
end
149+
150+
context "with no Gemfile.lock file in root, and no sub-dirs" do
151+
let(:target) { 'no_findings_no_gemfile_lock' }
152+
subject(:task_findings) { task.findings }
153+
it { is_expected.to eq([]) }
154+
end
155+
156+
context "with one Gemfile.lock in the root dir" do
157+
let(:raw_report) { get_raw_report(target) }
158+
159+
before do
160+
allow(task).to receive(:runsystem).with(*cli_args(target)).and_return(raw_report)
161+
task.run
162+
task.analyze
163+
end
164+
165+
context "with no findings" do
166+
let(:target) { 'no_findings' }
167+
subject(:task_findings) { task.findings }
168+
it { is_expected.to eq([]) }
169+
end
170+
171+
context "with one finding" do
172+
let(:finding) { task.findings.first }
173+
174+
context "of unknown criticality" do
175+
let (:target) { 'finding_2_unknown' }
176+
177+
it "has severity 2" do
178+
expect(finding.severity).to eq(2)
179+
end
180+
end
181+
182+
context "of low criticality" do
183+
let (:target) { 'finding_1' }
184+
185+
it "has severity 1" do
186+
expect(finding.severity).to eq(1)
187+
end
188+
end
189+
190+
context "of medium criticality" do
191+
let (:target) { 'finding_2' }
192+
193+
it "has severity 2" do
194+
expect(finding.severity).to eq(2)
195+
end
196+
end
197+
198+
context "of high criticality" do
199+
let (:target) { 'finding_3' }
200+
201+
it "has severity 3" do
202+
expect(finding.severity).to eq(3)
203+
end
204+
end
205+
end
206+
end
207+
end
208+
209+
end
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
#!/bin/bash
2+
# Runs bundle-audit on the contents of the 'targets' dir,
3+
# storing the output in 'report.txt' within each target folder.
4+
# Include a 'SKIP.txt' file next to 'package.json' if you don't want snyk to run on that target.
5+
6+
set -e
7+
8+
run_bundleaudit_recurs ()
9+
{
10+
if [ -f "Gemfile.lock" ] && [ ! -f "SKIP.txt" ]; then
11+
bundle-audit check > report.txt
12+
fi
13+
14+
for SUBTARGET in *
15+
do
16+
if [ -d ${SUBTARGET} ]; then
17+
cd ${SUBTARGET}
18+
run_bundleaudit_recurs
19+
cd ..
20+
fi
21+
done
22+
}
23+
24+
DIR=`dirname $0`
25+
cd "${DIR}/targets/"
26+
run_bundleaudit_recurs
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
GEM
2+
remote: https://rubygems.org/
3+
specs:
4+
kafo (0.3.1)
5+
6+
PLATFORMS
7+
ruby
8+
9+
RUBY VERSION
10+
ruby 2.3.3
11+
12+
BUNDLED WITH
13+
1.14.6
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
Name: kafo
2+
Version: 0.3.1
3+
Advisory: CVE-2014-0135
4+
Criticality: Low
5+
URL: http://osvdb.org/show/osvdb/106826
6+
Title: Kafo default_values.yaml Insecure Permissions Local Information Disclosure
7+
Solution: upgrade to ~> 0.3.17, >= 0.5.2
8+
9+
Vulnerabilities found!
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
GEM
2+
remote: https://rubygems.org/
3+
specs:
4+
http (0.7.1)
5+
6+
PLATFORMS
7+
ruby
8+
9+
RUBY VERSION
10+
ruby 2.3.3
11+
12+
BUNDLED WITH
13+
1.14.6
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
Name: http
2+
Version: 0.7.1
3+
Advisory: CVE-2015-1828
4+
Criticality: Medium
5+
URL: https://groups.google.com/forum/#!topic/httprb/jkb4oxwZjkU
6+
Title: HTTPS MitM vulnerability in http.rb
7+
Solution: upgrade to >= 0.7.3, ~> 0.6.4
8+
9+
Vulnerabilities found!
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
GEM
2+
remote: https://rubygems.org/
3+
specs:
4+
nokogiri (1.8.2)
5+
6+
PLATFORMS
7+
ruby
8+
9+
RUBY VERSION
10+
ruby 2.3.3
11+
12+
BUNDLED WITH
13+
1.14.6
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
Name: nokogiri
2+
Version: 1.8.2
3+
Advisory: CVE-2018-8048
4+
Criticality: Unknown
5+
URL: https://github.com/sparklemotion/nokogiri/pull/1746
6+
Title: Revert libxml2 behavior in Nokogiri gem that could cause XSS
7+
Solution: upgrade to >= 1.8.3
8+
9+
Vulnerabilities found!
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
GEM
2+
remote: https://rubygems.org/
3+
specs:
4+
curl
5+
6+
PLATFORMS
7+
ruby
8+
9+
RUBY VERSION
10+
ruby 2.3.3
11+
12+
BUNDLED WITH
13+
1.14.6

0 commit comments

Comments
 (0)