Commit graph

264 commits

Author SHA1 Message Date
Ben Johnson
ca29f7692b
Fix failing tests when IPv6 is disabled.
Turning off IPv6 causes the tests to fail because the Go
`net.Listen()` function obtains a `[::]` address when using
a `localhost:0` bind address. Looking through the Go source
code, I don't see a good way to fix that.

This change adds a test flag to allow users to specify the
network name when running the test:

```sh
go test github.com/pilosa/pilosa/server -network tcp4
```
2017-06-01 14:43:59 -06:00
Michael Baird
3b67f1bd92 return an error if the BroadcastHandler CreateFrameMessage encounters an Index that does not exist 2017-06-01 10:24:57 -05:00
Michael Baird
f0c3fdcdda Merge remote-tracking branch 'origin/master' into metrics 2017-05-30 15:36:19 -05:00
Michael Baird
f822bb43e9 Remove long query time from Server. That parameter is managed in the Cluster. 2017-05-30 12:55:45 -05:00
Matt Jaffee
b386fc9eac discriminate error messages in server.Open 2017-05-23 09:45:21 -05:00
Michael Baird
c41f695e74 more metrics tests 2017-05-18 16:36:22 -05:00
Linh Vo
4328cc88cc add nodeID tag 2017-05-17 16:23:34 -05:00
Michael Baird
7579567f59 return early when metrics are disabled 2017-05-16 15:28:18 -05:00
Michael Baird
4a051900c0 Add sampling rate to StatsD functions, and fix data dog logging 2017-05-16 10:35:49 -05:00
Michael Baird
bd44949cb5 Merge remote-tracking branch 'origin/master' into metrics
Conflicts:
	config.go
	executor_test.go
	server/server.go
2017-05-09 11:31:14 -05:00
Michael Baird
bf6ed942f0 Merge pull request #8 from alanbernstein/stats
Stats
2017-05-09 11:15:23 -05:00
Matthew Jaffee
0a81ce471d Merge pull request #524 from jaffee/ranked-default-2
change default cache type to ranked
2017-05-03 15:16:19 -05:00
Matt Jaffee
9fdb2e1460 differentiate error messages 2017-05-03 13:52:04 -05:00
Ben Johnson
e3d6f96657
Add max-writes-per-requests limit.
A configurable limit has been added to restrict the number of
mutating calls in a `pql.Query`. This is to prevents requests from
timing out from large queries.

The default is set to 5000 writes per request and is configurable
through the configuration file and the command line flags.
2017-05-03 08:53:20 -06:00
Michael Baird
ee159b455d Merge remote-tracking branch 'origin/master' into metrics
Conflicts:
	.gitignore
	cluster.go
	frame.go
	glide.lock
	index.go
2017-05-01 10:41:04 -05:00
Cody Soyland
6f571b9150 Merge pull request #502 from codysoyland/license
Apply Apache License 2.0
2017-04-28 22:17:08 -04:00
Cody Soyland
3996b56b88 Apply Apache License 2.0 2017-04-28 14:22:03 -05:00
Travis
28cbc621c2
set local node state to UP by default 2017-04-27 12:29:27 -05:00
Travis
7246e1e00d
Broadcast CreateSliceMessage when a new max slice is created 2017-04-27 11:44:13 -05:00
Michael Baird
6fa4c61061 pluralized owns slices function 2017-04-26 16:46:18 -05:00
Michael Baird
6a8e205769 Use OwnsSlice to determine the slice list per node and index for the Cluster Node Status. 2017-04-26 16:19:59 -05:00
Linh Vo
d18ee46d76 index, frame statsD and tests 2017-04-25 14:32:11 -05:00
Michael Baird
3512dd4549 missing comments 2017-04-25 14:15:02 -05:00
Michael Baird
e699c55700 fixed conflicts from the db index change
Conflicts:
	cluster.go
	executor_test.go
	handler_test.go
	server.go
2017-04-25 11:47:29 -05:00
Linh Vo
b450a62816 add entropy statsD 2017-04-25 09:22:30 -05:00
Michael Baird
0c9f652ffb missing comments from merge 2017-04-24 22:38:12 -05:00
Michael Baird
a543ba21bb fixed DB to index conflicts after merge 2017-04-24 21:59:39 -05:00
Michael Baird
5a8c165c53 Merge remote-tracking branch 'origin/master' into metrics
Conflicts:
	cache.go
	cluster.go
	db.go
	executor.go
	fragment.go
	frame.go
	gossip/gossip.go
	handler.go
	index.go
	server.go
	server/server.go
2017-04-24 16:35:25 -05:00
Travis
dfd17b38a4
Pass all Index/Frame meta data to other nodes via messages 2017-04-24 15:56:07 -05:00
Travis
74bf4731f5
WIP: Implement NodeState as an attribute of *Node
NodeState is shared among nodes in the cluster (via gossip
in a gossip implementation) and cached locally in Cluster.Nodes
in order to be available to the /status endpoint.
2017-04-24 12:33:43 -05:00
Travis
1343979308
Fixes up a few overlooked d := Index() 2017-04-24 11:17:30 -05:00
Travis
2ad322c2c7
Rename DB to Index
Rename `db.go` to `index.go` and `db_test.go` to `index_test.go`
2017-04-23 21:49:27 -05:00
Travis
2cacff8e09
Rename Index to Holder
Rename `index.go` to `holder.go` and `index_test.go` to `holder_test.go`
2017-04-23 15:59:20 -05:00
Alan Bernstein
6a162c4be2 Support LongQueryTime as config option 2017-04-21 18:50:25 -05:00
Alan Bernstein
ef8892ad01 Add versioned user-agent to requests 2017-04-21 10:15:58 -05:00
Michael Baird
940b62335c move the StateHandler interface from the Gossip package back to Pilosa and us this interface for the Handler to access the LocalState 2017-04-20 16:52:05 -05:00
Michael Baird
3f43b0d496 Merge remote-tracking branch 'travis/messenger-rebase' into metrics
# Conflicts:
#	server.go
2017-04-20 16:00:17 -05:00
Travis
3487bc373f
group Server interface implementation function together 2017-04-20 15:45:00 -05:00
Michael Baird
539b134e10 Store the node state information in Cluster
Access the overall cluster state info via the /status endpoint
2017-04-20 14:23:24 -05:00
Michael Baird
b0b6e2c544 measure runtime stats
added package to measure when garbage collection occurs
2017-04-19 17:22:40 -05:00
Travis
d2e68b2980
adjust some handler comments. remove debugging line 2017-04-19 17:08:10 -05:00
Matt Jaffee
2d139d299c remove server from handler - not used anymore 2017-04-19 10:36:27 -05:00
Matt Jaffee
bc8f7f702f add BroadcastReceiver and collapse Gossip structs into one
GossipNodeSet and GossipBroadcaster had a lot of cross dependency - I made them
the same object and had it implement all three interfaces (NodeSet, Broadcaster,
BroadcastReceiver).

I implmented an HTTPBroadcastReceiver which runs as as separate server.
BroadcastReceiver is a separate entity on the pilosa.Server object and is
started separately from broadcaster and nodeset. In the case of GossipNodeSet,
the broadcast receiver must be started before Open()ing the Nodeset, because it
doesn't actually start listening until Open() is called, but it needs the
handler set up before then.

server/server.go was heavily refactored to configure the new structs and
interfaces on the pilosa.Server object.
2017-04-19 10:36:27 -05:00
Matt Jaffee
32d07fc9e1 rename MessageBroker -> Broadcaster 2017-04-19 10:34:52 -05:00
Matt Jaffee
5079ee7a22 remove messenger "wrapper" around messagebroker
all the functionality aside from the broker has been removed
2017-04-19 10:33:59 -05:00
Matt Jaffee
1aca2ce1de implement LocalState and internal message reception on Server
greatly simplifies Messenger, to the point of making it basically shell around
MessageBroker. Next stesp are to make the receiving of internal messages more
well-defined and behind an interface, remove/merge Messenger and MessageBroker,
and have the implementations of MessageBroker in separate packages.
2017-04-19 10:33:26 -05:00
Travis
b0f1fc7523 Makes Messenger a first-class object under Server (with pointers in Handler and Index).
Primary message interface is the MessageBroker which is an attribute of the Messenger.
MessageBroker implementations:
- Gossip (memberlist)
- Broadcast (uses HTTP, received by existing Handler)
- Static (no-ops)

Changes CacheSize from `int` to `uint32` for consistency with protobuf.
Removes unnecessary dependencies in glide:
- `github.com/aws/aws-sdk-go`
- `golang.org/x/net` (although this gets included by memberlist)

TODO:
- [ ] Add tests around the Messenger and MessageBroker objects.
- [ ] Refactor CreateSliceMessage to work with views.
- [ ] Support propogation of meta data on PATCH calls.

fixing some issues from last rebase
2017-04-19 10:33:26 -05:00
Travis
cebca697b1 fixing tests that were affected by the messenger/view merge 2017-04-19 10:24:41 -05:00
Travis
8fa45649b6 WIP: Added support for NodeState.
Modified the gossip implementation to optionally send messages directly (instead of over gossip).
Added support for local and remote NodeState, as well as function to merge remote into local state.
Refactored Protobuf models to include `DBMeta` and `FrameMeta` as well as support for NodeState.
2017-04-19 10:19:17 -05:00
Travis
2b35df879c WIP: first pass at sending CreateDB() through Messenger 2017-04-19 10:15:57 -05:00