Roaring likes to call things "a" and "b", not "1" and "2",
and use "n" for length, not "l", etcetera. Adopt these
conventions to make code more readable.
Also drop the 'vb' value since it isn't expensive to
compute and the compiler can figure out that it can
reuse the value.
confirm that the number of runs comes out as expected,
and add a couple of numbers out of order to verify that
the 17-18-19 set gets coalesced into one run even
if we add 17 and 19 before 18.
The net effect of this is to not recompute "the current
value of the first array" on every loop, pretty much.
However, the swap to make sure the inner loop is on the
longer array seems to be significant for performance.
On my system, this moves runtime from ~29us per op
to ~17us per op.
If the total number of things returned was small enough to
make an array, intersectBitmapRun converted to an array. This
seems possibly-premature; future processing might well prefer
a bitmap. We know everything gets optimized before being
written out, let's not convert without a specific reason. But
also, let's use an array no matter which container is small
enough to prove that we can do so safely.
Fixes#854.
The circumstances under which bitmaps are converted between
types are not 100% nailed down, and the IntersectionCount
benchmark was actually using a bitmap for the "run" data set
as well as for the "bitmap" data set. Fix that by using
Optimize() explicitly. Also, add a second RLE set so we
can compare the difference between "one run for the entire
set" and "several runs".
Also add array/array comparisons. We use two different
lengths of arrays, because performance turns out to vary
between "first array longer" and "second array longer".
Also added a benchmark for getBenchData itself, since it's
at least one possible use case for "creating a lot of
containers".
bitmapEquals isn't currently being called ever, but it has
an arcane edge-case bug, so I've made the test case for it
and commented it out for future reference.
This commit adds some trivial stat-tracking which can be
observed at localhost:10101/debug/vars. However, writes to
a locking data structure aren't cheap, so the stat-tracking
is by default not compiled. To build it, add the build
tag `roaringstats`, which will cause the `statsHit` function
to actually do something. Otherwise, it's an empty and
inlineable function, meaning the compiler throws it away
entirely.
This would, in principle, let us get additional visibility
into edge cases and which code paths are hot. This is not
the same thing as profiling for overall performance; the
stat counts aren't affected by whether a particular code path
is using a large amount of CPU time, just reporting how
often it happens at all.
I'd like to add stat tracking to Roaring, which means it
has to be able to import the stats package, which means
stats has to be a package rather than part of the pilosa
package. If stats stops being in pilosa, it still needs
a way to import logger, so logger also has to leave the
pilosa package. Then everything using them needs to import
them and use package selectors on their names.
This doesn't actually add the stats support to roaring,
it just makes it so there's a way to import the stats
code from something in the roaring package.
