Commit graph

107 commits

Author SHA1 Message Date
Todd Gruben
4a1d15ec9d missed a file on last commit 2018-05-04 10:48:57 -07:00
Todd Gruben
2ef78af0ca adjusted protection around cluster coordinator mutations 2018-05-04 10:48:57 -07:00
Matthew Jaffee
992c3e8bdd WIP - adding more locking to Cluster 2018-05-04 10:48:57 -07:00
Matthew Jaffee
f9ff20689a
remove holder.Peek, combine with HasData, move server logic
Server initializing happens more in NewServer than Open now - expecting to
continue this trend. goal was to remove remoteClient from Server (since it has a
defaultClient) as well, but we'll have to refactor the client usage in fragment
and frame first.
2018-04-24 08:44:58 -05:00
Matthew Jaffee
876ed56e30
move pilosa.Config to pilosa/server.Config
step 1 of #1203

The Config object is really just a specification of the options to pilosa
server, so it makes sense to have it in that package.
2018-04-19 14:51:42 -05:00
Travis Turner
0a8d573fb3
WIP: Remove SecurityManager. Implement api restrictions in api package. 2018-04-16 17:12:28 -05:00
Matthew Jaffee
3f303d1098
remove global defaults from config.go
these were occaisionally referenced elsewhere in the codebase - in all but one
case, there were workarounds that are actually better I think.

In the one case there wasn't I created a single top level DefaultConfig object
which is instantiated with all the default values and can be referred to if
necessary.

There was a bug in fragment.go with the way MaxWritesPerRequest was treated if
it was 0. Elsewhere, 0 meant no limit, but here, it would have caused a division
by 0.

Changed the default metrics provider from "nop" to "none", although "nop" will
still work. Previously, any value other than "statsd" or "expvar" was treated as
"nop", but I've changed this behavior to return an error if an invalid string is
provided. I think this is better behavior, because in the case that someone
bothered to change the default, they were probably interested in actually
getting stats, and might be annoyed when it silently failed.
2018-04-12 19:52:48 -05:00
Matthew Jaffee
04f5d5875c
api docs, rename funcs, refactor usage of internal
All exported funcs in api.go are now documented

Several poorly named methods of API and Cluster were renamed. Particularly, the
word Fragment was often changed to Slice in cases where it was really a slice
being specified and not a fragment.

several methods which received or returned internal data structures have been
refactored to be more opaque.

Deprecation logging was added to input definition methods.
2018-04-12 11:28:35 -05:00
Matthew Jaffee
f8cb579187
get tests passing 2018-04-09 14:29:43 -05:00
Travis Turner
9a2aeac6f8
Clean up logger; make it honor --log-path flag.
Add functional options to NewGossipMemberSet.
2018-03-27 08:40:04 -05:00
Travis Turner
2a462d5e42
make sure cluster.Nodes[].IsCoordinator values get updated. return old coordinator node in response. 2018-03-08 14:18:10 -06:00
Travis Turner
fa4e543e84
send a NodeJoin event on startup for cases where a quick restart has occurred and memberlist is not aware of it 2018-03-08 12:42:24 -06:00
Travis Turner
1cc45b22a2
Change Config.Coordinator from URI to bool 2018-03-08 12:42:24 -06:00
Cody Soyland
5a2e1c3703
Merge pull request #1162 from codysoyland/cluster-resize-remove-without-replicas
Proper error handling when attempting to remove node when there aren't enough replicas
2018-03-07 13:51:44 -06:00
Cody Soyland
42682e12a8 Address code review: Fix error handling and add comment 2018-03-07 09:45:27 -06:00
Cody Soyland
b7b92913d9 Add comment to listenForJoins 2018-03-06 15:13:24 -06:00
Cody Soyland
d28a30ebd4 Proper error handling when attempting to remove node when there aren't enough replicas 2018-03-06 15:11:46 -06:00
Travis Turner
d0009206b4
add comments to exported methods. remove debugging test. 2018-03-06 12:42:29 -06:00
Travis Turner
29dcfd06da
Merge pull request #1117 from travisturner/cluster-resize-handle-error
replace swallowed error with log entry
2018-02-23 16:37:42 -06:00
Travis Turner
52edeb72b4
add validation around node-remove conditions 2018-02-22 12:15:27 -06:00
Travis Turner
c4f6a6beda
Merge branch 'master' into cluster-resize 2018-02-21 16:23:53 -06:00
Travis Turner
666fe22ecb
replace swallowed error with log entry 2018-02-15 12:11:27 -06:00
Travis Turner
346e92a91d
return 0 values for errors. panic on unmarshal node meta data 2018-02-02 15:58:51 -06:00
Travis Turner
86dbbbf393
don't require oldNode in SetCoordinator() 2018-02-02 15:58:51 -06:00
Travis Turner
216ba7a41e
Use NodeID instead of URI for node identification 2018-02-02 15:58:46 -06:00
Travis Turner
878b188155
add accessor method for Node.status 2018-01-24 17:03:58 -06:00
Travis Turner
88fa51e6bf
add lock around Node.status to avoid race condition 2018-01-24 16:03:36 -06:00
Travis Turner
f12527d535
put a mutex around Cluster.State 2018-01-23 16:00:32 -06:00
Travis Turner
e57c125c08
getting some minor fixes out of my stash 2018-01-04 10:45:04 -06:00
Travis Turner
6b3f4d1a64
Merge branch 'master' into cluster-resize
Refactored LocalID/ClusterID for use with Topology and Coordinator.
2017-12-18 15:39:50 -06:00
Travis Turner
0342da92bb
WIP: add tests for cluster resize
This commit adds support for allocating a gossip transport
and a server listener prior to opening server (and cluster).
Doing that allows tests to use a dynamically allocated port
by supplying bind port: 0.
2017-12-12 16:27:47 -06:00
Travis Turner
cf1e41faf1
Merge branch 'master' into cluster-resize 2017-12-08 15:22:12 -06:00
Todd Gruben
e6adb361b9 cleanup logging 2017-12-07 15:09:42 -06:00
Todd Gruben
9565db43a8 add sort order to URI addition 2017-12-07 14:11:49 -06:00
Todd Gruben
c14bc29041 added logging for node membership 2017-12-07 11:17:37 -06:00
Todd Gruben
f17a399a37 adjust to single http client 2017-12-07 08:33:26 -06:00
Travis Turner
073848ae31
Make sure node gets removed from all nodeSets after resize 2017-12-04 18:03:23 -06:00
Travis Turner
59cdd5dde9
Add HolderCleaner and view.DeleteFragment to support post-resize cleanups
Add tests for view.DeleteFragment and HolderCleaner
2017-12-04 18:03:22 -06:00
Travis Turner
8228ef2382
Add a Cluster.Static override for tests to treat static nodes as
Coordinator.
Refactor the Server.joining channel to be in Cluster instead.
2017-11-28 16:40:05 -06:00
Travis Turner
ccdd6262c5
add error logging for non-topology nodeJoin 2017-11-28 12:43:49 -06:00
Travis Turner
ae63adfaac
let remote nodes know that its safe to open Holder when launching based on existing taxonomy 2017-11-27 11:06:03 -06:00
Travis Turner
013cd0cd95
refactor resize instruction logic to support multi-index. wait to open holder on non-coordinator nodes. 2017-11-27 11:06:03 -06:00
Travis Turner
f2c32f8ec9
add logging support to ResizeJob 2017-11-27 11:06:03 -06:00
Travis Turner
bd511dae80
ensure that holder opens before node is deemed ready 2017-11-27 11:06:03 -06:00
Travis Turner
a13db570cc
WIP: Wait for nodeState on Holder.Open().
Implement prefect in Cluster so a node can start http listener in a
restricted mode.
Adjust tests; particularly start test.Holder in state Normal.

TODO:
- [ ] fix test TestMain_SendReceiveMessage
- [ ] add additional tests for `nodeState`
2017-11-27 11:06:03 -06:00
Travis Turner
a98b862fca
add /cluster/resize/remove-node endpoint 2017-11-27 11:06:02 -06:00
Travis Turner
426992bc59
add set-coordinator endpoint 2017-11-15 09:13:52 -06:00
Michael Baird
59035883f4 group the write operations in syncBlock by MaxWritesPerRequest 2017-11-10 15:03:23 -06:00
Travis Turner
9d87019762
WIP: don't block joining nodes while coordinator loads data.
Implements a Holder.Peek() function, and breaks out
Cluster.ListenForJoins() into a separate method that can be started
after the Holder finishes loading data.

TODO:
- [ ] test Holder.Peek()
2017-11-10 09:14:13 -06:00
Travis Turner
e641aa1d8a
Merge branch 'master' into 'cluster-resize' 2017-11-07 15:00:11 -06:00