Commit graph

286 commits

Author SHA1 Message Date
Travis
cad78f1d34
remove "Default*" from const names 2021-03-05 16:56:44 -06:00
Nia Weiss
4340e90396
start HTTP handler after server initialization
This fixes a variety of bugs where API requests would read uninitialized state, causing crashes or race conditions.

Co-authored-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-02-26 15:23:47 -05:00
Seebs
4f5f3e30ea remove port_mapper because it can't work with our unrestartable server
Long story short: Once we create a server and start it, we can't start
it again. We can't close it and restart it, and we can't just start
it without closing it.

Unfortunately, if the server's config needs to change, we have a Problem
here.

This ultimately means that the retry logic for GetListeners can't actually
retry successfully; if we fail on the first attempt, we necessarily fail
on any later attempts also, and if we try to fix that, we get panics.

But!

We don't actually NEED to retry. We just need to ensure that we can
open a :0 port, extract the actual port number, and use that in places
where the port number mattered, without having to rebind it.

The only actual place we needed to rebind things was opening gRPC
servers, so we introduce a gRPC Listener that can be used instead of
trying to bind to a specified port.

In a bunch of other cases where we had similar logic to try to allocate
and then use a port, we can switch to just using a provided listener.
For instance, net/http has `Serve(net.Listener, handler)`, not just
ListenAndServe(addr, handler).

This should eliminate the weird CI failures from eaddrinuse.

NOT fixed: server/cluster_test.go/TestClusterResize_AddNode isn't working
right now. The new node isn't actually being added to the existing cluster.
I attempted this but was outsmarted by it, and I think fixing the
rest of this is worth it as a separate thing.
2021-02-24 11:25:46 -06:00
Travis
5b237cae13
Merge branch 'master' into disco 2021-02-12 20:29:41 -06:00
Maxton Huff
b89dae01cc clarify purpose of trialVersion by changing the name to handleTrialDeadline and adding a doc string 2021-02-11 16:13:04 -06:00
Maxton Huff
dd8d6ebfb1 Respond to code review feedback
change logic in version.go so that the trial related messages only appear on trial versions of molecula
convert Command methods in trial.go to functions and  pass a loggerLogger variable instead since that was the only piece of Command being used
add a function named expireAfter which seperately runs similar functionality to what was previously in daily check with chnages directed at stopping users from changing their internal clock date
change variable names and placement to be more readable and organized
2021-02-11 11:01:16 -06:00
Maxton Huff
c60893a8ee edit and move trial code from server.go to trial.go and combine release build targets 2021-02-11 11:01:16 -06:00
Maxton Huff
7fd3fc775f remove test code 2021-02-11 11:01:16 -06:00
Maxton Huff
d02ba32743 alter error output to satisfy test 2021-02-11 11:01:15 -06:00
Maxton Huff
9840ae67e4 remove error return type 2021-02-11 11:01:15 -06:00
Maxton Huff
1dbab1ff92 add draft trial version of molecula 2021-02-11 11:01:15 -06:00
Antonio Navarro Perez
5440e177de Use advertised URL
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-02-11 11:52:42 +01:00
Antonio Navarro Perez
1f234df86e Really fix url on Web UI log
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-02-11 11:10:38 +01:00
Travis
a4b37273ea
remove the rest of the gossip code (except config) 2021-02-04 13:03:02 -06:00
Travis
652014539c
remove temporary Gossiper interface 2021-02-03 23:06:51 -06:00
Antonio Navarro Perez
c45e21640c
Change coordinator to primary
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-02-02 15:56:12 -06:00
Travis
26176c15eb
fix linter issues (wrap all ClusterStates in string() until we update the type) 2021-02-01 16:57:35 -06:00
Kuba Podgórski
6058fc22e4
Porting disco.Stator (next step) 2021-02-01 16:35:59 -06:00
Travis
97eaff5c82
use Etcd Noder; actually use EtcdWithCache 2021-01-31 23:42:44 -06:00
Travis
457194f6a8
update config to support etcd arguments 2021-01-29 19:43:48 -06:00
Travis
e459c9a77b
change Config.DisCo to Config.Etcd 2021-01-29 14:11:51 -06:00
Travis
1c0b926eae
Merge master into disco 2021-01-28 18:03:38 -06:00
Kuba Podgórski
cef6925e7b Fix server tests 2021-01-28 17:39:58 +01:00
Maxton Huff
927db378b4 add linux OS check and the way mmap limit is read 2021-01-27 09:52:13 -06:00
Maxton Huff
3982a8e970 format messages and change mmap comparison logic 2021-01-26 11:26:36 -06:00
Maxton Huff
3809fe6734 add error check 2021-01-26 09:35:51 -06:00
Maxton Huff
55a9952b92 mmap limit comparison error message 2021-01-25 16:19:12 -06:00
Travis
a196e1e74c
use etcd for node.ID
this commit adds a temporation interface for starting gossip.
we needed this so we can start gossip AFTER setting up the node,
but before waitingForJoins.
2021-01-23 19:52:58 -06:00
Travis
13984353e4
remove instances of os.Getenv("PILOSA_TXSRC") 2021-01-21 21:18:46 -06:00
Travis
f292d6061a
replace pilosa.DefaultTxsrc with storage.DefaultBackend 2021-01-20 22:06:13 -06:00
Travis
08fae2be4c
introduce storage.Config 2021-01-20 22:05:38 -06:00
Jason E. Aten
b1a0e0ae8b allow retry at the cluster level to work; remove retry for server/server.go Command.setupNetworking() that retries a single gossip node 2021-01-18 20:39:23 +00:00
Jason E. Aten
ed12f3585c etcd/embed.go implemented Noder 2021-01-18 20:30:21 +00:00
Todd Gruben
de7d69234e initial listner 2021-01-18 10:34:18 -06:00
Kuba Podgórski
201e851511 set more ports for Node Command 2021-01-14 20:04:09 +01:00
Antonio Navarro Perez
052aadb3b0 Wrap some missing constructors using ports.
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-01-14 10:48:46 +01:00
Travis
27614c42f7
Finish implementing port wrapper 2021-01-13 22:56:54 -06:00
Antonio Navarro Perez
d20b831084 Add port wrapper POC
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-01-13 20:14:33 +01:00
Travis
bc13834343
disco/etcd work: fix lots of races, start all cluster nodes at once.
port mapper gives out ports from 63000-65000 for the tests

fix another race

http test uses port.MustGetPort

rbf: remove :0 port request

ocd happy

test fix for grpc listener address already in use

test/disco allocates BindGRPC port from the port mapper

dump stack on each GetPort

verify each port is usable right away

server/config.go has Config.Validate() now

panic if gossip port is 0. validate server.Config

fix another gossip port 0

builds

quiet, don't dump stack on each port alloc

builds

happy linter

even gossip fallback should not be zero but rather use the port mapper
2021-01-12 21:06:12 -06:00
Travis
20816ffa20
remove pilosa.URI 2021-01-06 16:19:14 -06:00
Travis
4515a24e48
change all references to use subpackages: topology, net 2021-01-06 16:09:24 -06:00
Maxton Huff
dee700741a deprecate cluster.long-query-time and create long-query-time
moved lonquerytime from cluster into server and moved cluster.longquerytime into top level config
kept cluster.longquerytime for backwards compatibility, favored if both longquerytime options are present
2020-12-10 10:51:31 -06:00
Jason E. Aten
03a54c6d5f rowcache off by default. pilosa server --rowcache-on turns it back on. 2020-12-08 23:18:45 +00:00
Nia Weiss
7ea2e4b7e3
add ID auto-generation on the coordinator 2020-12-03 13:04:14 -05:00
Alan Bernstein
2bc2a32263 Make query history length configurable 2020-11-23 20:23:27 -06:00
Jason E. Aten
957cba1768 performance tuning: rbfcfg package, binary search for wal segment
- rbfcfg package holds Config for --rbf- command line flags
- wal.go: replace linear search with bisection for wal segment
2020-10-27 00:49:56 +00:00
Jason E. Aten
8d2ad048fa pilosa server --norowcache disables the row cache
- this can lessen memory pressure
- certain backends may not need it
- enables performance benchmarking and tuning
2020-10-22 22:20:48 +00:00
Cody Soyland
31cd119a7e Move grpc.Server creation into initializer
This fixes an issue with the grpc-web middleware using a nil
grpc.Server instance.
2020-10-16 11:36:41 -05:00
Cody Soyland
04b1152224 Add grpc-web to http handler 2020-10-16 11:36:41 -05:00
Jason E. Aten
fe425a84c0 pilosa-fsck: scan and repair of pilosa backups
- the -fix flag repairs replication errors by copying from the primary.
 - the -fixkeys flag repairs any string key translation issues.
 - make pilosa-fsck installs pilosa-fsck and builds release-pilosa-fsck.COMMIT.GOOS.tar.gz release tarbar
2020-10-02 16:47:56 -05:00