We create a lot of these during a large GroupBy query or anything else
that creates a ton of filters. Use a pool so we can reuse them, since
most of their data doesn't need to be zeroed out, and typical use
patterns have a lot of sequential creation of these short-lived things
within a goroutine.
We don't really need to fully extract every row, we just need counts.
This naive approach uses logic similar to BitmapBitmapFilter, but tweaks
it so that we can intercept the existence and sign bit rows, work with
an optional filter, and yield a sum. We accumulate the statistics
internally, rather than using a callback, because I tried to make it
work with a callback and it was a complete mess.
Note the fancy check for container reuse in the BSI Count filter.
This is because intersection(full container, X) is just the original
X, *not* a copy, but in this case we need a copy because RBF
ApplyFilter will in fact reuse a single container's storage for
each consecutive container.
We used to manually do this because we had a number of cases where
BitN wasn't being updated, but so far as we know we've fixed them
and we have run a fair amount of stuff with sanity checks on and
not hit anything, so eliminating the constant recounting on bitwise
containers seems like a win.
We have some tests that cover stuff like the DumpDot functionality,
but we don't need them to actually write to stdout during ordinary
testing. Dump to buffers which we politely ignore. Yes, we could have
used a dummy writer, but this way it's super easy to display the
contents if we find ourselves suddenly caring.
When closing, we need to wait for existing Tx to exit before truncating
files and unmapping things. This shouldn't matter, because we don't actually
close the DB until all transactions are done, normally... except for the
background usage-gathering task. But really, it's probably just better to
be conservative.
The actual logic is fancier than it looks. We can't hold db.mu.Lock during
this, or the existing Tx can't exit. So we first grab the lock, set the closed
flag, set up a waiter for all current Tx to exit, and then release the lock.
Now we wait on the current Tx exiting. Once that's done, we grab the locks.
Anything coming in that tries to start a Tx will fail out fairly quickly
because the opened flag is now false, so even if other things get those
locks before we do, they won't keep them or create new Tx.
This makes one test deadlock because it opens a Tx and never closes it,
so we change that test to close its Tx.
Check if queries that have a 'like' argument are applied to keyed
fields. If not, log that the user is trying to use 'like' on an
unsupported field type (as opposed to reporting that there
are no results.)
This should help prevent a data race. SetBit can, in some cases, cause an
asynchronous task to run which tries to update the stats counter.
But if that task runs while we are modifying the stats counter itself, we have a
data race.
Add a lock to the Server WaitGroup so that if the Server WaitGroup is already
waiting, we won't concurrently add to it and cause a data race.
Also, when adding to the Server WaitGroup, check that the server is not closing
already, since that means we really shouldn't be doing more work.
When deletion is started, _exists field is updated with row+1.
After deletion is completed, we delete _exists=row+1.
If _exists>=1, then deletion was not completed.
Updated go version in docker to match other requirements.
Removed duplicate error check for grpc.
