de14762661
CountRange for RBF had a subtle bug which wasn't noticed, so, let's have some CountRange testing and also a benchmark. We also fix a couple of subtle bugs caught in the process of developing and testing this. SliceContainers will allow nil containers, but doesn't return them when iterating because there's various things that can panic if called on a nil container. Since countEmptyContainers() has to traverse the whole bitmap anyway, it doesn't matter which it counts, so we replace it with countNonEmptyContainers(), and adjust test cases accordingly. This fixes an issue where if roaring is smart enough to insert a nil container into a SliceContainers, trying to write it to a file produces an invalid bitmap with offsets off by 16 and one container fewer than its header predicts. RBF: don't try to count 0 bits in a container If we're to the "last container", and we'd be counting all the bits less than zero, we can skip that. This avoids hitting a bug, which is that c.countRange doesn't handle BitmapPtr. |
||
|---|---|---|
| .. | ||
| benchpretty | ||
| testdata | ||
| add.go | ||
| add_test.go | ||
| btree.go | ||
| btree_test.go | ||
| container_archetypes.go | ||
| container_stash.go | ||
| containers_btree.go | ||
| containers_slice.go | ||
| containers_test.go | ||
| filter.go | ||
| filter_internal_test.go | ||
| fuzz_test.go | ||
| fuzzer.go | ||
| generation_debug.go | ||
| generation_nodebug.go | ||
| inst.go | ||
| naive.go | ||
| naive_test.go | ||
| nop_inst.go | ||
| printutil.go | ||
| printutil_test.go | ||
| README.md | ||
| roaring.go | ||
| roaring_container_test.go | ||
| roaring_helpers_test.go | ||
| roaring_internal_test.go | ||
| roaring_nop_paranoia.go | ||
| roaring_nop_sentinel.go | ||
| roaring_nop_stats.go | ||
| roaring_paranoia.go | ||
| roaring_sentinel.go | ||
| roaring_stats.go | ||
| roaring_test.go | ||
| source.go | ||
| unmarshal_binary.go | ||
The Fuzzer
For complete documentation on go-fuzz, please see: https://github.com/dvyukov/go-fuzz
The fuzzer in relation to the roaring package checks the Bitmap.UnmarshalBinary function found in roaring.go. In order to use the fuzzer, you can follow these steps:
cd $GOPATH/src/github.com/pilosa/pilosa/roaring
go-fuzz-build ./
You must now make the workdir/corpus directory. This is achieved by:
mkdir workdir/corpus
The fuzzer needs some input to start the fuzzing with. Copy some sample Pilosa fragments into the workdir/corpus folder. For example:
cp ~/.pilosa/my-index/my-field/views/standard/fragments/0 workdir/corpus
Once you have copied your sample inputs, you are ready to run the fuzzer:
go-fuzz -bin=roaring-fuzz.zip -workdir=workdir -func=FuzzBitmapUnmarshalBinary
Understanding the Fuzzer Output
The fuzzer will output something similar to the follwoing:
2015/04/25 12:39:53 workers: 8, corpus: 124 (12s ago), crashers: 37, restarts: 1/15, execs: 35342 (2941/sec), cover: 403, uptime: 12s
The most important part of the output is the crashers and cover. The crashers records how many combinations were discovered that fail and the cover tells you how much code is being accessed.
For a complete explanation of the output, please see: https://github.com/dvyukov/go-fuzz.
The fuzzer will document the crashers in a folder labeled "crashers." It will record the fragment and the error that was produced in two separate files within this folder. This is the final product.
Happy Fuzzing!