Attributes are unmaintained and unused.
They have become more of a liability than a benefit.
This change eliminates them from the codebase.
The only user-visible change (assuming that attrs are not used) is that the attrs field will no longer appear in row JSON.
We want to distinguish different *kinds* of GroupCounts, so we're
making the GroupCounts parent object track its type so we can keep that
correct.
Adding this to protobuf, etc, then creates some weird behaviors
because sometimes we expect []GroupCount, and sometimes we expect
*GroupCounts. This implies changes to test cases. Also, the
changes to test cases imply that some test cases are probably now
wrong; for instance, they're expecting a "sum" column, equal to zero,
when no sum was requested.
We try to make the encoder handle a []*GroupCount gotten from another
node without panicing, and avoid breaking the semantics of the existing
messages, renumbering messages or components, etc.
Since a previous version, the `.Groups` member has been privatized,
and the `.Get()` convenience accessor has been renamed `.Groups()`
and is now used consistently in a way that should reduce the risk
of nil pointers causing crashes. Also, NewGroupCounts is used in
a couple more places.
port mapper gives out ports from 63000-65000 for the tests
fix another race
http test uses port.MustGetPort
rbf: remove :0 port request
ocd happy
test fix for grpc listener address already in use
test/disco allocates BindGRPC port from the port mapper
dump stack on each GetPort
verify each port is usable right away
server/config.go has Config.Validate() now
panic if gossip port is 0. validate server.Config
fix another gossip port 0
builds
quiet, don't dump stack on each port alloc
builds
happy linter
even gossip fallback should not be zero but rather use the port mapper
We execute the aggregate Distinct calls after the GroupBy is complete,
and we need these to act like non-remote calls in that they forward to
all nodes, but like remote calls in that they bypass key
translation. Added a "PreTranslated" flag to the QueryRequest to
achieve this.
Discovered an issue where a nil *Row in EmbeddedData would cause a
panic in the protobuf serialization. Changed the encoding code we
control to never pass a nil *Row.
Got fed up with lack of context on errors and added wrapping to all
calls under executor.executeCall as well as a few other places.
Handled a situation where not having data on a shard for a particular
field could cause a query to error instead of just treating that
fragment as being empty. (see the switch in executeDistinctShardSet)
Stopped GroupBy from executing the Count(Distinct) aggregate on Remote
calls.
Fixed a longstanding issue where errors retrieved from remote query
calls had a garbage character at the front due to treating a protobuf
payload as an error message instead of decoding it. (see
http/client.go)
This commit changes executeDistinct to return either a *Row or a
SignedRow (instead of only being able to return a SignedRow). Distinct
on a set field will return a *Row while an int field will still return
a signed row.
We then add Field and Index fields to the Row object so that we can
determine how to translate the rows IDs to keys (if needed). This adds
a lot of logic around the translation which fixes bugs where Distinct
calls would fail to get translated.
There are, I think, still issues if you were to try to join a keyed
field to a keyed index which wasn't explicitly specified as the
field's foreign index. The IDs in the field wouldn't be using the same
translation as the IDs in the index, so the query might appear to work
but give incorrect results.
- Atomic record contains multiple ImportRequest and
ImportValueRequest, plus ability to Clear individual requests.
- adds http handlers for importing AtomicRecord.
We want to be able to control whether or not we use roaring to
serialize Rows, which means serializers have to be able to be
distinct.
We also make corresponding changes to http/handler.go to have
it use the exported serializers directly rather than the API's
serializer (which is always the base protobuf serializer
right now, and if it weren't, that would be bad because we
were assuming it was).
When we're accepting protobuf from a pilosa server, flag that
we'll accept roaring bitmaps as opposed to the naive column
representation.
I think this will improve the transaction response messages Kuba
mentioned where it was an empty transaction instead of a nil or not
there... if not it should make it easier to do that anyhow.
also adds a "noSleep" option to the server command to avoid the 5
second sleep we introduced on startup for non-coordinator cluster
nodes. The sleep doesn't seem to be needed in the tests and makes them
much slower.
This commit adds `TranslationSources` to the cluster
`ResizeInstruction`. These are the sources of translation
partitions which the receiving node needs in order to support
partition distribution in the new, resized cluster.
This also fixes a bug where index options were not being
encode in the proto Index object. That meant that the schema
transferred via protobuf was not correct. The reason why
things normally worked is because index creation typically
happens on the CreateIndex message, which does include the
options.
TODO:
- [ ] implement the TranslateStore interface for `InMemTranslateStore`
and `mock.TranslateStore`
- [ ] surely need some more tests around the `ReadFrom` and `WriteTo`
This PR adds support for anti-entropy syncing for integer
and decimal fields. It differs from the logic for other
field types in that it does not rely on a consensus to determine
what the value should be; instead, it considers the correct
values to be those of the primary replica. From there, data
is pushed to all non-primary replicas.
this involved adding an optional float value to the ValCount struct
which complicated result types, necessitated grpc changes, and needed
quite a few tests at different layers.
I ran:
brew upgrade protobuf
GO111MODULE=off go get -u github.com/gogo/protobuf/protoc-gen-gofast
I'm not sure if everything is still going to work, but I'm excited to
find out!
This allows a BSI field to have an option indicating
that it is a foreign key to another index. If the foreign
index has column keys, then this field handles string values
by using the foreign index's translate store.
This PR adds a field name (string) to the return types
which represent the values from a specific field. For example,
a TopN query on field `x` would be `TopN(x)` and have results
like:
```
[]Pair{
{ID: 14, Count: 10},
{ID: 3, Count: 8},
{ID: 7, Count: 3},
}
```
In order to know what field this result type refers to, we wrap
`[]Pair` in a new struct called `PairsField` which contains an
addition `Field` string where `x` is stored.
This is useful for informing the gRPC server how to construct
more appropriate headers for the result stream (in this case,
the column headers can now be "x" and "count").
Similar logic was applied to `RowIdentifiers` and `Pair` as well.
