Depending on where in the replicate() loop you are when a
store is closed or reassigned, it's possible for it to deadlock.
The deadlock would be that replicate has just successfully read an
entry from your PrimaryTranslateStore.Reader, when a new
PrimaryTranslateStore event happens. Then handlePrimaryTranslateStore
grabs the mutex, signals that the replication handler should
close, and waits for the replication handler to close. Meanwhile,
the replicate() loop tries to grab the mutex... and deadlocks.
Solution: Make the replicate() loop part that needs the mutex
a goroutine that signals on a channel, so we can put it in a select
along with checking for the replicationClosing signal (or the
context terminating). If one of those happens, replicate()
terminates, allowing monitorReplication() to return, which
causes the anonymous function which called it to call
repWG.Done(), allowing handlePrimaryTranslateStore to continue
and eventually release the mutex. At some later point, appendEntry
succeeds or fails, dumps its result status in a buffered
channel, and exits, and the buffered channel is garbage collected.
This is way simpler than it sounds, but it took me a while
to figure out how simple it was.
This provides a simple benchmark that can be used for
setValue, to give a way to compare results from adding BSI
support to roaring. Use the BSIGroup prefix for the
fragments, and specify a cache type of "none", to prevent
the use of a LRU cache (which makes things more expensive).
Add a parallel benchmark for ImportValue, so we can compare
them. (Unsurprisingly, the bulk-import endpoint is quite a
lot faster.)
Also, add a test for clearing values to the TestFragment_Sum
test; it turns out that this was never tested in this code,
but the http client test would test it and verify it, it should
probably also be tested here.
