port mapper gives out ports from 63000-65000 for the tests
fix another race
http test uses port.MustGetPort
rbf: remove :0 port request
ocd happy
test fix for grpc listener address already in use
test/disco allocates BindGRPC port from the port mapper
dump stack on each GetPort
verify each port is usable right away
server/config.go has Config.Validate() now
panic if gossip port is 0. validate server.Config
fix another gossip port 0
builds
quiet, don't dump stack on each port alloc
builds
happy linter
even gossip fallback should not be zero but rather use the port mapper
Add ability to sort on count or aggregate in GroupBy. Fix bug with offset being unsupported. Fix bugs with limit interacting poorly with other arguments.
the limit could get applied before "having" in some cases which could
result in results being discarded which met the having condition while
results were kept which did not, ultimately resulting in GroupBy
falsely reporting fewer results than actually existed.
Back out support for sorting on fields (only count and aggregate
supported for now).
Fix bug where default return of "true" caused sort to be unstable. (If
they are equal, Less should return false)
Fix bug where limit was being applied before sorting.
Fix bug where offset was not actually allowed to be an argument to
GroupBy (weird! guess we weren't testing that very well)
Apply "having" after calculating Count(Distinct) aggregate so that
having can apply to that.
Switch to stable sort to make testing easier.
We execute the aggregate Distinct calls after the GroupBy is complete,
and we need these to act like non-remote calls in that they forward to
all nodes, but like remote calls in that they bypass key
translation. Added a "PreTranslated" flag to the QueryRequest to
achieve this.
Discovered an issue where a nil *Row in EmbeddedData would cause a
panic in the protobuf serialization. Changed the encoding code we
control to never pass a nil *Row.
Got fed up with lack of context on errors and added wrapping to all
calls under executor.executeCall as well as a few other places.
Handled a situation where not having data on a shard for a particular
field could cause a query to error instead of just treating that
fragment as being empty. (see the switch in executeDistinctShardSet)
Stopped GroupBy from executing the Count(Distinct) aggregate on Remote
calls.
Fixed a longstanding issue where errors retrieved from remote query
calls had a garbage character at the front due to treating a protobuf
payload as an error message instead of decoding it. (see
http/client.go)
If you're running a Pilosa with mostly default configuration on your
system, some of these tests would fail due to things like port
conflicts. These changes address the most common failures.
From Nia:
While debugging the Q2 bugs this was somewhat useful in analyzing cluster events. As for the spammy part. . . that seems to be more of an issue with spamming our resets than an issue with the log itself.
This used to be possible to hit, but I think now that Distinct on a
set field returns a *Row rather than a SignedRow it isn't an issue. (I
wasn't able to trigger it in the tests). Adding the fix anyway as it
seems safer than not.
The rest of the changes are test infrastructure to make it easy to
call GRPC queries and verify the results as CSV.
I used a "paranoia" check to find these, but then realized the check
had a ton of false positives and doing it properly wasn't going to be
straightforward. I'm leaving the paranoia stuff in unless there are
objections, because I've wanted it before and not had it.
I also removed a log line that is very verbose and I don't think helps
anyone.
This commit changes executeDistinct to return either a *Row or a
SignedRow (instead of only being able to return a SignedRow). Distinct
on a set field will return a *Row while an int field will still return
a signed row.
We then add Field and Index fields to the Row object so that we can
determine how to translate the rows IDs to keys (if needed). This adds
a lot of logic around the translation which fixes bugs where Distinct
calls would fail to get translated.
There are, I think, still issues if you were to try to join a keyed
field to a keyed index which wasn't explicitly specified as the
field's foreign index. The IDs in the field wouldn't be using the same
translation as the IDs in the index, so the query might appear to work
but give incorrect results.
