In the case where a translate log entry contained
many key/id pairs, it was possible for the read
buffer (which was allocated at 65536 bytes) to
fail to handle it. This happened when the serialized
LogEntry was larger than 65536 bytes.
This PR adds logic which returns a custom error called
ErrTranslateReadTargetUndersized notifying the reader
to reallocate a larger read buffer and try the read
again.
TODO:
- [ ] Add a max buffer size check to prevent this from doubling the
buffer size with no limit.
- [ ] Add tests.
suspect that this is somehow causing "cannot assign requested address" bugs for
some users. removing since it wasn't a necessary part of the deadlock fix, but
just seemed like a prudent thing to have.
I'd like to add stat tracking to Roaring, which means it
has to be able to import the stats package, which means
stats has to be a package rather than part of the pilosa
package. If stats stops being in pilosa, it still needs
a way to import logger, so logger also has to leave the
pilosa package. Then everything using them needs to import
them and use package selectors on their names.
This doesn't actually add the stats support to roaring,
it just makes it so there's a way to import the stats
code from something in the roaring package.
Long ago, the maintainers of gorilla/mux concluded that
it was a "wontfix" to return StatusMethodNotAllowed instead
of StatusNotFound for a method mismatch. Pilosa had a
workaround for this for the most common case (GET requests
to /index/{index}/query), and a TODO to address the other
cases.
While browsing the go-pilosa client code, I noticed that there
is a test for roaring import support which relies on getting
StatusMethodNotAllowed. But how can this work, if gorilla/mux
doesn't do that?
Answer: They started doing it in mid-2017, apparently:
https://github.com/gorilla/mux/issues/271
Dropping this code changes the exact text of the message
produced for that one case, but not the status code,
and makes the behavior less confusing.
This commit adds the ability to remove an 'available shard'
from the shard cache. This does not affect shards known to be
available because of local data.
