featurebase/roaring
Seebs e7e3331fb4 test intersectionCallback more carefully
This takes our reasonably broad selection of predefined container
types and tries intersectionCallback on each pair of them, comparing
results against the results of plain old intersect(). We've had
several intersectionCallback fixes recently; every one of them
produces test failures here if reverted or broken, so I have at
least some confidence in this coverage.

Similarly, test everything on containerCallback, verifying that
we get the same set of values called back that we get from Slice().

Both of these were verified with -coverprofile to actually be
hitting all the lines of code that aren't insane edge case
checks like "what if a run is in the wrong order".
2021-09-16 14:05:35 -05:00
..
benchpretty FeatureBase Renaming: changing go.mod module name for featurebase 2021-07-19 09:20:30 -07:00
testdata cleanup #1622 2018-09-06 16:27:10 -05:00
add.go implement TopK on time 2020-11-20 11:06:55 -05:00
add_test.go skip roaring add test in race 2020-11-20 12:16:09 -05:00
btree.go roaring: fix use-after-free in b-tree bitmap update 2020-05-20 12:01:21 -04:00
btree_test.go enforce append-like semantics for *Container more consistently 2020-09-28 12:19:24 -05:00
container_archetypes.go container performance benchmarking 2020-10-22 14:33:03 -05:00
container_stash.go rbf/intoContainer: ensure correct N, avoid recounting 2021-04-12 16:13:16 -05:00
containers_btree.go enforce append-like semantics for *Container more consistently 2020-09-28 12:19:24 -05:00
containers_slice.go when updating a container, drop the single-container cache 2020-08-17 11:23:47 -05:00
containers_test.go all test green on rbf. WOOT. 2020-08-12 21:08:30 +00:00
filter.go make details optional and support limits on mutex checks 2021-09-08 11:59:47 -05:00
filter_internal_test.go make details optional and support limits on mutex checks 2021-09-08 11:59:47 -05:00
fuzz_test.go generalize test strings and break out old UnmarshalBinary code 2019-08-05 17:39:47 -05:00
fuzzer.go blue_green migration. holdbkg.go holder goroutine. 2020-09-11 14:24:15 -05:00
generation_debug.go Sources and Generations: tracking mmapped files 2019-11-12 12:14:29 -06:00
generation_nodebug.go Sources and Generations: tracking mmapped files 2019-11-12 12:14:29 -06:00
inst.go Add license headers to files missing them and CI check to verify they are present. Fixes #1633 2019-04-12 11:30:41 -05:00
naive.go added go-fuzz testing for roaring ops vs naive implementation 2019-06-25 10:47:53 -05:00
naive_test.go Lower scale of some random-value tests 2020-06-08 12:10:40 -05:00
nop_inst.go double the nolint comments, double the checking 2019-10-11 15:17:51 -05:00
printutil.go FeatureBase Renaming: changing go.mod module name for featurebase 2021-07-19 09:20:30 -07:00
printutil_test.go FeatureBase Renaming: changing go.mod module name for featurebase 2021-07-19 09:20:30 -07:00
README.md Fixed typo 2019-06-17 16:47:28 -05:00
roaring.go record stats for intersectionCallback under the right name 2021-09-16 14:05:35 -05:00
roaring_container_test.go roaring: fix intersectionAnyRunBitmap when processing single-word runs 2021-01-21 18:09:27 -05:00
roaring_helpers_test.go all test green on rbf. WOOT. 2020-08-12 21:08:30 +00:00
roaring_internal_test.go test intersectionCallback more carefully 2021-09-16 14:05:35 -05:00
roaring_nop_paranoia.go Check more carefully for, and also fix, containers with invalid N 2020-10-14 10:03:18 -05:00
roaring_nop_sentinel.go switched naive_test.go to table driven tests 2019-06-25 17:25:07 -05:00
roaring_nop_stats.go Add license headers to files missing them and CI check to verify they are present. Fixes #1633 2019-04-12 11:30:41 -05:00
roaring_paranoia.go Check more carefully for, and also fix, containers with invalid N 2020-10-14 10:03:18 -05:00
roaring_sentinel.go switched naive_test.go to table driven tests 2019-06-25 17:25:07 -05:00
roaring_stats.go FeatureBase Renaming: changing go.mod module name for featurebase 2021-07-19 09:20:30 -07:00
roaring_test.go FeatureBase Renaming: changing go.mod module name for featurebase 2021-07-19 09:20:30 -07:00
source.go Sources and Generations: tracking mmapped files 2019-11-12 12:14:29 -06:00
unmarshal_binary.go Prototype ingest API 2021-08-19 09:50:59 -05:00

The Fuzzer

For complete documentation on go-fuzz, please see: https://github.com/dvyukov/go-fuzz

The fuzzer in relation to the roaring package checks the Bitmap.UnmarshalBinary function found in roaring.go. In order to use the fuzzer, you can follow these steps:

cd $GOPATH/src/github.com/pilosa/pilosa/roaring

go-fuzz-build ./

You must now make the workdir/corpus directory. This is achieved by:

mkdir workdir/corpus

The fuzzer needs some input to start the fuzzing with. Copy some sample Pilosa fragments into the workdir/corpus folder. For example:

cp ~/.pilosa/my-index/my-field/views/standard/fragments/0 workdir/corpus

Once you have copied your sample inputs, you are ready to run the fuzzer:

go-fuzz -bin=roaring-fuzz.zip -workdir=workdir -func=FuzzBitmapUnmarshalBinary

Understanding the Fuzzer Output

The fuzzer will output something similar to the follwoing:

2015/04/25 12:39:53 workers: 8, corpus: 124 (12s ago), crashers: 37, restarts: 1/15, execs: 35342 (2941/sec), cover: 403, uptime: 12s

The most important part of the output is the crashers and cover. The crashers records how many combinations were discovered that fail and the cover tells you how much code is being accessed. For a complete explanation of the output, please see: https://github.com/dvyukov/go-fuzz.

The fuzzer will document the crashers in a folder labeled "crashers." It will record the fragment and the error that was produced in two separate files within this folder. This is the final product.

Happy Fuzzing!