A configurable limit has been added to restrict the number of
mutating calls in a `pql.Query`. This is to prevents requests from
timing out from large queries.
The default is set to 5000 writes per request and is configurable
through the configuration file and the command line flags.
It is nested under "cluster" in the config, and it controls cluster membership
as well as broadcasting, so I think type is more appropriate. Also, brevity.
GossipNodeSet and GossipBroadcaster had a lot of cross dependency - I made them
the same object and had it implement all three interfaces (NodeSet, Broadcaster,
BroadcastReceiver).
I implmented an HTTPBroadcastReceiver which runs as as separate server.
BroadcastReceiver is a separate entity on the pilosa.Server object and is
started separately from broadcaster and nodeset. In the case of GossipNodeSet,
the broadcast receiver must be started before Open()ing the Nodeset, because it
doesn't actually start listening until Open() is called, but it needs the
handler set up before then.
server/server.go was heavily refactored to configure the new structs and
interfaces on the pilosa.Server object.
greatly simplifies Messenger, to the point of making it basically shell around
MessageBroker. Next stesp are to make the receiving of internal messages more
well-defined and behind an interface, remove/merge Messenger and MessageBroker,
and have the implementations of MessageBroker in separate packages.
moved into pilosa/server/server.go with minimal changes for now - had to tweak a
few things to be able to get at unexported fields. Next step will be to move the
implementations of NodeSet and MessageBroker out of the pilosa package so that
pilosa core knows nothing about them. pilosa/server will deal with any
complexities involved in setting them up.
Primary message interface is the MessageBroker which is an attribute of the Messenger.
MessageBroker implementations:
- Gossip (memberlist)
- Broadcast (uses HTTP, received by existing Handler)
- Static (no-ops)
Changes CacheSize from `int` to `uint32` for consistency with protobuf.
Removes unnecessary dependencies in glide:
- `github.com/aws/aws-sdk-go`
- `golang.org/x/net` (although this gets included by memberlist)
TODO:
- [ ] Add tests around the Messenger and MessageBroker objects.
- [ ] Refactor CreateSliceMessage to work with views.
- [ ] Support propogation of meta data on PATCH calls.
fixing some issues from last rebase
Modified the gossip implementation to optionally send messages directly (instead of over gossip).
Added support for local and remote NodeState, as well as function to merge remote into local state.
Refactored Protobuf models to include `DBMeta` and `FrameMeta` as well as support for NodeState.
1. fixed a couple bugs with config file reading.
2. made cmd.Serve a global variable so tests could inspect it.
3. added table style testing for pilosa server
4. added the ability to stop server programmatically which also causes the cobra
command running it to return.
Now that the default data dir is defined by the flag, we don't need to set it
explicitly if it isn't set. We also stop reading the config file explicitly in
server.go since viper will read it - we do need to define all the config options
though before the config file will work properly.
