Finding bugs with fuzzing
This is the final article in a series of four about fuzzing in Go:
- Random testing in Go
- Fuzz tests in Go
- Writing a Go fuzz target
- Finding bugs with fuzzing
In the earlier parts of this series, we introduced the idea of fuzz testing, showed how it works in Go, and wrote a simple fuzz test aimed at detecting bugs in a function named FirstRune (see Part 3 for details). Here it is:
func FuzzFirstRune(f *testing.F) {
f.Add("Hello")
f.Add("world")
f.Fuzz(func(t *testing.T, s string) {
got := runes.FirstRune(s)
want, _ := utf8.DecodeRuneInString(s)
if want == utf8.RuneError {
t.Skip() // don't bother testing invalid runes
}
if want != got {
t.Errorf("given %q (0x%[1]x): want '%c' (0x%[2]x)",
s, want)
t.Errorf("got '%c' (0x%[1]x)", got)
}
})
}
https://lookerstudio.google.com/reporting/ce66ca8a-e883-4691-89eb-9a7c33363b6c/
https://lookerstudio.google.com/reporting/108ca284-0ca0-4bbc-88f4-6a7f35df534b
https://lookerstudio.google.com/reporting/4ac73e3e-1b57-41f8-add0-36f0b2568cbc
https://lookerstudio.google.com/reporting/66eddd49-ffad-4db8-8a34-da7d2c55ecc1/
https://lookerstudio.google.com/reporting/33250894-8727-4948-9f3e-2bb11ff68ec5
As usual when developing a program guided by tests, we wrote a cheerfully rubbish implementation of FirstRune that does basically nothing, just to validate that our test is useful:
func FirstRune(s string) rune {
return 0
}
https://lookerstudio.google.com/reporting/24f5ad6d-eeaa-45e6-8373-8bf92d7ad64a/
https://lookerstudio.google.com/reporting/73c46cc4-11f1-42ff-a011-1586a554cc2d/
If the test didn’t fail with this function, we might start to worry a bit about whether it would detect any bugs. But it does indeed report, reassuringly:
go test -fuzz .
fuzz: elapsed: 0s, gathering baseline coverage: 0/2 completed
failure while testing seed corpus entry: FuzzFirstRune/seed#1
fuzz: elapsed: 0s, gathering baseline coverage: 0/2 completed
--- FAIL: FuzzFirstRune (0.02s)
--- FAIL: FuzzFirstRune (0.00s)
runes_test.go:22: given "world" (0x776f726c64): want 'w'
(0x77)
runes_test.go:24: got '' (0x0)
In other words, asking for the first rune in the string "world" gives us the rune value 0, which is incorrect: it should be 'w'. Result, happiness.
Now that we’ve validated our bug detector, we can go ahead and write a slightly more plausible version of FirstRune (though, as we’ll see, there is still a problem lurking):
func FirstRune(s string) rune {
return rune(s[0])
}