This was in response to some feedback we got about the new release
format. Executables were no longer had executable permission due to
going through S3 (hence the tarballs), and we wanted a more consistent
directory structure in the final release which included the versions
of various components.
this is related to work for [fb-1127](https://molecula.atlassian.net/browse/FB-1127)
cardinality reporting has caused no shortage of issues such that we recommend
disabling them almost everywhere.
this commit removes the cardinality calculation for right now, as well as the option
to enable/disable schema details.
If a field doesn't exist, looking up that field produces a nil,
and querying the name of a nil field fails. Don't do that. Instead,
just use the name you're looking it up by.
We could in theory return an error here, but we already handle
nonexistent fields elsewhere and checking this when we already have
checks for it seems unnecessary, I think?
Also, we add a test for this. The test is over in server/grpc_test.go
because we have infrastructure there for testing the SQL server
functionality, and you can't actually write reasonable self-contained
tests for the SQL stuff because it has no way to create a working
server.
We create a lot of these during a large GroupBy query or anything else
that creates a ton of filters. Use a pool so we can reuse them, since
most of their data doesn't need to be zeroed out, and typical use
patterns have a lot of sequential creation of these short-lived things
within a goroutine.
We don't really need to fully extract every row, we just need counts.
This naive approach uses logic similar to BitmapBitmapFilter, but tweaks
it so that we can intercept the existence and sign bit rows, work with
an optional filter, and yield a sum. We accumulate the statistics
internally, rather than using a callback, because I tried to make it
work with a callback and it was a complete mess.
Note the fancy check for container reuse in the BSI Count filter.
This is because intersection(full container, X) is just the original
X, *not* a copy, but in this case we need a copy because RBF
ApplyFilter will in fact reuse a single container's storage for
each consecutive container.
We used to manually do this because we had a number of cases where
BitN wasn't being updated, but so far as we know we've fixed them
and we have run a fair amount of stuff with sanity checks on and
not hit anything, so eliminating the constant recounting on bitwise
containers seems like a win.
We have some tests that cover stuff like the DumpDot functionality,
but we don't need them to actually write to stdout during ordinary
testing. Dump to buffers which we politely ignore. Yes, we could have
used a dummy writer, but this way it's super easy to display the
contents if we find ourselves suddenly caring.
When closing, we need to wait for existing Tx to exit before truncating
files and unmapping things. This shouldn't matter, because we don't actually
close the DB until all transactions are done, normally... except for the
background usage-gathering task. But really, it's probably just better to
be conservative.
The actual logic is fancier than it looks. We can't hold db.mu.Lock during
this, or the existing Tx can't exit. So we first grab the lock, set the closed
flag, set up a waiter for all current Tx to exit, and then release the lock.
Now we wait on the current Tx exiting. Once that's done, we grab the locks.
Anything coming in that tries to start a Tx will fail out fairly quickly
because the opened flag is now false, so even if other things get those
locks before we do, they won't keep them or create new Tx.
This makes one test deadlock because it opens a Tx and never closes it,
so we change that test to close its Tx.
