This is a unification of a number of bug fixes, feature additions,
and so on. Features include:
* Dropping the "New" from NewWrite/NewRead.
* IndexName->keys.Index, etc.
* Add a new "Flush" operation which is necessary to get the
intended behavior of Delete, which allows us to commit/flush
changes without letting go of a write lock.
* Some additional wrapping and locking in rbfTxWrappers to
support that. rbfQueryRead/Write now forward their calls
to the parent rbfTxWrappers, so it can lock around the
reference to its underlying tx, so the flush operation can
replace that tx safely.
* AddIndexShards now treats no shards as "all shards", to
simplify call sites.
* Added parameters to NewRBFTxStore to let it interact with
executor's logger and worker pool.
* Internally, support explicit closes of parts of the database
which can also check for errors and fail if it's in use.
* Add ability to request a map of fields and views in use
for a given index/shard pair. This is probably deprecated
but we need it for the way backup/restore work.
* Add ability to request a complete map of the database showing
which shards exist for which index/field/view tuples. This is
backwards from how we store things on disk, but we need it
to allow creating the right in-memory data structures on
database open.
* Add "Backend()" method to let us distinguish backends in case
we some day have them again.
* Support deleting indexes, fields, or fragments.
* Support Backup (returning a ReadCloser that dumps the RBF
file, implicitly merging any current WAL) and Restore (create
a new RBF file).
* Change directory structure and fragment keys to match existing
databases, so we should in theory be able to open an existing
data directory.
* Fragment delete probably doesn't lock correctly and this
should be reviewed.
* Export the DOT-format Dump so we can hook it up to a debug
endpoint. This wants to be explored more; ideally the front-end
UI should be able to display this.
* Create a NopTxStore which can be used like a TxStore but everything
that can error errors out. This is then used to let a holder that
hasn't had a txstore initialized work anyway.
There's at least a couple of open issues that need to be revisited
here.
This provides us with most of the existing Tx interface, split
across QueryRead and QueryWrite. The functions not included here
are the ones that are used *only* for anti-entropy (ForEach
and ForEachRange).
We add additional testing to verify that TxStores are getting
closed correctly, to go with cleaning up the test directories they're
made in.
We also introduce some test wrappers that can automatically
fail tests on error, so tests don't need to be full of error
checks.
Also, now that I'm starting to think more about the flow of
writing tests using QueryScope, we add the missing "full
database" scope option, and make the Add methods return
their operand so (1) you can chain them, (2) you can use
the AddIndex(...) inline in a NewWriteQueryContext.
Also addressed a plausible performance concern in shardList,
and some comments that were stale or incorrect.
The test coverage here is skimpy on the actual RBF-calling
functions because those are trivial. We do, however, significantly
expand coverage in the random write requests, which are now
a mix of random writes and random reads, and add test cases
that at least hit a lot of the error checks once.
The Error() method is changed to be like (testing.T).Error(),
taking ...interface{} and using fmt.Sprint on them.
There's also some minor tweaks such as making the visualizations
more consistent, testing visualization generation on two kinds
of keysplitter, and so on.
