This commit adds `roaring.Bitmap.CountRange()` to return the number
of bits in a subrange of a bitmap. This significantly improves
server start time and is needed for upcoming zero copy bitmap
optimizations.
This commit moves the cache flush to the `Index` and only serializes
a single fragment at a time.
Also included in this commit is the `inspect` command for the
`pilsoactl` binary. This provides insight into pilosa data files.
This commit refactors the block checksumming by removing the
iteration over each block and instead only checking blocks which
have data. This requires merging the cache inspection with the
roaring iterator to reduce CPU time and memory allocations.
This commit adds a `Closing` channel to the `IndexSyncer` and the
`FragmentSyncer` that is periodically checked during execution.
If the channel is closed then an in-progress sync is immediately
stopped.
This commit adds a fast path for optimizing queries that only
have `SetBitmapAttrs()` calls in them. It does this by grouping
the calls into a single Bolt transaction and also by merging
multiple attribute updates per ID into one update.
This commit changes the `roaring.Bitmap` to use a 40/24 split in
the high and low bits. This change also requires the low bits
to use `uint32` instead of `uint16`.
This commit moves bitmap intersection into the `roaring` package
in order to reduce allocations.
Benchmarks against real world datasets show a speed improvement of
353%. Previously, intersection operated at approximately 5.6M
bits/sec against two 60K bit bitmaps with 30% overlap. The new
intersection operates at approximately 20M bits/sec.
This commit changes the fragment allocation algorithm in the cluster
to make use of the `DB` name. This allows each database to use a
different slice distribution.
Initially, the `frame` was going to be used for allocation, however,
this was problematic since queries can span multiple frames so it's
impossible to choose a single frame to use.
