also adds a "noSleep" option to the server command to avoid the 5
second sleep we introduced on startup for non-coordinator cluster
nodes. The sleep doesn't seem to be needed in the tests and makes them
much slower.
This commit adds `TranslationSources` to the cluster
`ResizeInstruction`. These are the sources of translation
partitions which the receiving node needs in order to support
partition distribution in the new, resized cluster.
This also fixes a bug where index options were not being
encode in the proto Index object. That meant that the schema
transferred via protobuf was not correct. The reason why
things normally worked is because index creation typically
happens on the CreateIndex message, which does include the
options.
TODO:
- [ ] implement the TranslateStore interface for `InMemTranslateStore`
and `mock.TranslateStore`
- [ ] surely need some more tests around the `ReadFrom` and `WriteTo`
This allows a BSI field to have an option indicating
that it is a foreign key to another index. If the foreign
index has column keys, then this field handles string values
by using the foreign index's translate store.
This commit adds a Decimal field type which is implemented mostly with
the Int field. It adds an optional "Scale" value to the Int field
which means that the values stored in that field are actually meant to
be divided by 10^Scale before being interpreted.
In order to make use of this functionality, we extend the importValue
request to allow a slice of floats rather than just int64. If the
slice of floats is present, each float in the slice is multiplied by
10^Scale and converted to an int64 before being imported. If a slice
of int64 is imported to a Decimal field, it is treated normally, and
scale is ignored. This allows the conversion to be handled at the
client side if desired.
Currently there are Field level methods for querying Float values out
of a decimal field, but no support in PQL or the executor for getting
float values. Going to wait until I can use the generic result type
before doing that, so for now, any values queried will be the scaled
integer values.
needed to add client support for importing float values, and did this
by adding a more general and simplified client method for value
imports.
rewrote api.ImportValue to use the new method which should be more
performant and efficient.
allow floats to be "pilosa import"ed into decimal fields
This commit implements BSI with variable bit depth using a
sign magnitudeto indicate whether a value is positive or negative.
This also rearranges the existence bit to be the first bit instead
of the last bit.
There was a situation where availableShards on a new
node were not in sync with the cluster, so queries
following a resize were incorrect.
- Start a one-node cluster.
- Write data to shards 0 and 1
- Start a second node.
In the case where the hash algo was moving shard 0 to
node1, then node1 only knew about shard 0, so queries
to node1 would be incomplete.
This PR modifies the ResizeInstruction message to replace
`Schema` with `NodeStatus` (which contains both `Schema` and
`AvailableShards`). So now when a resize instruction is received,
the receiving node is able to sync its schema and availableShards.
This commit adds the ability to remove an 'available shard'
from the shard cache. This does not affect shards known to be
available because of local data.
This commit removes the previous `MaxShard` tracking and replaces
it with an `Available Shards` set tracking. This allows sparse shard
tracking without implicitly tracking all shards in between.
This commit adds support for allocating a gossip transport
and a server listener prior to opening server (and cluster).
Doing that allows tests to use a dynamically allocated port
by supplying bind port: 0.
Implement prefect in Cluster so a node can start http listener in a
restricted mode.
Adjust tests; particularly start test.Holder in state Normal.
TODO:
- [ ] fix test TestMain_SendReceiveMessage
- [ ] add additional tests for `nodeState`
