Commit graph

320 commits

Author SHA1 Message Date
Matt Jaffee
5079ee7a22 remove messenger "wrapper" around messagebroker
all the functionality aside from the broker has been removed
2017-04-19 10:33:59 -05:00
Matt Jaffee
1aca2ce1de implement LocalState and internal message reception on Server
greatly simplifies Messenger, to the point of making it basically shell around
MessageBroker. Next stesp are to make the receiving of internal messages more
well-defined and behind an interface, remove/merge Messenger and MessageBroker,
and have the implementations of MessageBroker in separate packages.
2017-04-19 10:33:26 -05:00
Travis
b0f1fc7523 Makes Messenger a first-class object under Server (with pointers in Handler and Index).
Primary message interface is the MessageBroker which is an attribute of the Messenger.
MessageBroker implementations:
- Gossip (memberlist)
- Broadcast (uses HTTP, received by existing Handler)
- Static (no-ops)

Changes CacheSize from `int` to `uint32` for consistency with protobuf.
Removes unnecessary dependencies in glide:
- `github.com/aws/aws-sdk-go`
- `golang.org/x/net` (although this gets included by memberlist)

TODO:
- [ ] Add tests around the Messenger and MessageBroker objects.
- [ ] Refactor CreateSliceMessage to work with views.
- [ ] Support propogation of meta data on PATCH calls.

fixing some issues from last rebase
2017-04-19 10:33:26 -05:00
Travis
cebca697b1 fixing tests that were affected by the messenger/view merge 2017-04-19 10:24:41 -05:00
Travis
8fa45649b6 WIP: Added support for NodeState.
Modified the gossip implementation to optionally send messages directly (instead of over gossip).
Added support for local and remote NodeState, as well as function to merge remote into local state.
Refactored Protobuf models to include `DBMeta` and `FrameMeta` as well as support for NodeState.
2017-04-19 10:19:17 -05:00
Travis
2b35df879c WIP: first pass at sending CreateDB() through Messenger 2017-04-19 10:15:57 -05:00
Travis
7c2f6a9bae adds a basic gossip implementation using hashicorp/memberlist
Changes Gossiper to NodeSet.
Adds StaticNodeSet (for testing) and GossipNodeSet (for memberlist) implementations.
Changes NodeSet interface to return a pilosa-specific generic instead of `NodeSet interface`.
Removes NumMembers from interface (which is specific to memberlist).

Implements a Messenger interface with which to send inter-node messages via NodeSet.
The Pilosa implementation occurs in the GossipNodeSet.

Implements the Messenger as an object on Server, Handler, and Index.

Uses HealthStatus constants.
Removes commented-out code.

for gossip, make sure to bind to both host and port, and advertise those as well

adjust messenger to work with the db schema logic

add dependencies: hashicorp/memberlist, golang.org/x/sync
add dependency: golang.org/x/net

Uses `errgroup` to handle errors from broadcast messages.
Marshals message one time instead of once for every node.
Adds error handling for some errors that were being swallowed.
2017-04-19 10:13:38 -05:00
Matt Jaffee
5f267c0a3a support the various ways of specifying bind addr 2017-03-17 16:43:04 -05:00
Ben Johnson
7de0293034 Add basic schema support.
Databases and frames now require explicit creation and have the
option of setting row & column labels. If no labels are provided
then the default `id` and `profileID` labels are used.
2017-02-20 13:55:03 -07:00
jaffee
ffcb4c0c10 set Index.LogOutput to be Server's as well as handler 2017-01-18 09:01:01 -06:00
Travis
659db70a57 Fixes merge conflicts related to MaxSlices.
Merge branch 'time' of github.com:benbjohnson/pilosa into benbjohnson-time
2016-12-12 09:04:23 -06:00
Ben Johnson
c6e2b1294b Add time-based frames and Range() support. 2016-12-08 12:29:49 -07:00
Travis
dd6f41f551 standardizes on MaxSlice (instead of SliceN)
creates DB locally even if node doesn't have data for that DB
2016-11-30 12:59:17 -06:00
Travis
78666d6bba Merge branch 'master' into db-specific-slices 2016-11-29 11:59:41 -06:00
Cody Soyland
5206461d51 Migrate from Umbel to Pilosa organization on Github 2016-11-28 15:21:11 -06:00
Travis
93e3590737 set DefaultPollingInterval back to its original value 2016-11-21 16:36:17 -06:00
Travis
b6f0a62fdd changes MaxSlices logic to be DB-specific 2016-11-21 16:12:30 -06:00
Ben Johnson
d5ad5fbb05 Add profile & bitmap attribute anti-entropy.
Adds block-based anti-entropy for the attribute stores and hooks
into the existing `IndexSyncer` to regulate frequency of syncs.
2016-09-30 11:47:43 -06:00
Ben Johnson
521b098c83 check for early close during AAE
This commit adds a `Closing` channel to the `IndexSyncer` and the
`FragmentSyncer` that is periodically checked during execution.
If the channel is closed then an in-progress sync is immediately
stopped.
2016-06-29 10:18:24 -06:00
Ben Johnson
a0a519bfdf refactor main into pilosa.Server
This commit refactors most of the code in `cmd/pilosa` to
`pilosa.Server` so that it can be reused in long running cluster
testing.
2016-05-13 14:39:09 -06:00