This greatly simplifies the code, and with the recent addition of DirectAddN and
DirectRemoveN should be as or more performant than doing the separate bitmap and
union (in most cases, unsorted data could still be slower). Perhaps more
importantly, it is also less allocation heavy than the union approach. Also
makes it trivial to get the counts of changed bits, so I've cleaned up the stats
to show number of bits we're importing/clearing and the number of bits that
actually changed.
I'd like to add stat tracking to Roaring, which means it
has to be able to import the stats package, which means
stats has to be a package rather than part of the pilosa
package. If stats stops being in pilosa, it still needs
a way to import logger, so logger also has to leave the
pilosa package. Then everything using them needs to import
them and use package selectors on their names.
This doesn't actually add the stats support to roaring,
it just makes it so there's a way to import the stats
code from something in the roaring package.
move rowFilters to fragment.go
new mergeGroupCounts implementation takes limit into account while merging,
exploits inherent order of group count results.
fragment panics if rowsForColumn is called with a column id not in the
fragment's shard. The justification for this is that we're wasting resources if
we're sending requests for a specific column to any shard other than the one
which contains that column.
pass previous+1 directly to fragment.rows so that the iterator can seek directly
to the start point. handle limit inside reduce so it can skip out early and
avoid extra allocation.
there isn't much piont in making snapshot take a WriterTo, because the only
thing that's going to be written to a fragment is a roaring bitmap. making it
WriterTo just makes it slightly more annoying to jump into the Bitmap.WriterTo
implementation. since it's a private function, it should be straightforward to
change this in the future if the need arises.
