When unmarshalling ops, we weren't adding a meaningful OpN to them,
resulting in misleading reports from `pilosa inspect`. Also, we were
mistakenly reporting things as "mapped" when they were actually
using their internal storage (as with small array containers).
Add the "sanity check" to `pilosa inspect` so that errors like the
above get noticed more easily and corrected. Also, to make that work,
have roaring.InspectBinary actually put containers in the bitmap
it creates rather than just creating info entries for them.
bitmap.BitwiseEqual had a couple of subtle bugs, and the net result
is that if the bitmap you were comparing to had an empty container after
the original bitmap ran out of containers, we'd spuriously report
the container as existing and being... the last container in the original,
actually.
Issues are both that we were grabbing the value from the wrong iterator,
and also that we were iterating twice per loop, and thus could also
have missed a non-empty container immediately following an empty one.
When *fragment.openStorage is invoked in both f.importValue and
f.importValueSmallWrite and it returns an error, this means there's
some underlying error with the storage device and at the point of this
commit, the sane thing to do is to close the process, otherwise the
operation of Pilosa might proceed in an inconsistent state thus
precipiatting other silent but hairy errors along the way such as
dereferencing *fragment.gen later on which is set to nil once
openStorage fails.
This changes Count(Precall()) operations to execute the precall directly inside of the count operation, bypassing the transformation to a Precomputed() call.
Eliminating the Precomputed() step causes Count(Distinct()) to work properly on negative integers.
We need to be able to count bits in BitmapPtr containers. This only
comes up if you have a non-container-aligned range count, which we
never do in real production yet, but the API allows it so it should
work. In order to do this, we need to provide the tx to countRange
so it can grab pages as needed. Arguably, we should be able to avoid
actually creating/copying that page since we're only using it
internally, never returning it, but this is a pretty rare case
and probably not performance-critical.
This test was supposed to check against all the container types,
but especially bitmaps, but turns out not to work because the
containers turn into RLE containers. Oops. Now, we start with
every-other-bit for the first 8k, then start filling in the holes,
so we get some bitmap containers and then start generating
RLE containers.
