scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-08-28 10:54:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
7473 commits 45 branches 174 tags 85 MiB
Commit graph

406 commits

Author SHA1 Message Date
Todd Gruben
94e66b74b5 tls config bug and arge handling 2021-05-24 11:00:37 -05:00
Todd Gruben
42b465b80c load schema 2021-05-21 09:27:08 -05:00
Todd Gruben
9d24fb07b7 wired in restore command 2021-05-21 09:27:08 -05:00
Todd Gruben
7c6423587a skeleton restore 2021-05-21 09:27:08 -05:00
Ben Johnson
a4f282c8e8 Allow backup to stdout 2021-05-19 15:04:36 -06:00
Ben Johnson
776b43a3cd Backup CLI 2021-05-07 10:59:01 -06:00
Alan Bernstein
285d0a0af8 Add log prefix levels 2021-04-12 20:33:39 -05:00
Kuba Podgórski
9a02004a4f Move vprint to separate pakage 2021-03-29 14:29:19 +02:00
Matt Jaffee
11288c2ae8
Revert "Basic pilosa changes for oracle support" 
This reverts commit 47e74f2603.
 2021-03-14 22:17:26 -05:00
Seebs
1045268f01 use testhook to ensure temporary files and directories are cleaned up 
In nearly all cases, we can just switch ioutil.TempDir->testhook.TempDir
and similarly for TempFile. There's one case where we can't because we
need files to be removed before tests are over.

Also in the process give identifiable names to a lot of temporary files
and make sure they're being cleaned up, and don't use "/tmp/foo" as a
file name in a test that could be running in more than one test process
at once. :)
 2021-03-11 19:42:10 -06:00
tgruben
21910ef321
Update main.go 2021-03-11 11:47:57 -06:00
tgruben
a5dceaae11
Update main.go 2021-03-11 11:26:58 -06:00
Todd Gruben
47e74f2603 Basic pilosa changes for oracle support 2021-03-11 11:16:53 -06:00
Travis
2bbe1fdde0
remove remaining references to "coordinator" 2021-02-23 17:23:09 -06:00
Travis
a2a6e91f6d
remove old, now conflicting test value 2021-02-12 21:21:56 -06:00
Travis
114f6a8751
add withViews argument to api.Schema() method 2021-02-08 10:42:55 -06:00
Travis
4dd6bddf7f
remove pilosa-chk 2021-02-05 16:13:43 -06:00
Travis
c8c59b649d
remove pilosa-fsck 2021-02-04 21:34:15 -06:00
Travis
a4b37273ea
remove the rest of the gossip code (except config) 2021-02-04 13:03:02 -06:00
Antonio Navarro Perez
87ba73fa16 Stop writes on DEGRADED state 
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
 2021-02-04 17:30:14 +01:00
Travis
457194f6a8
update config to support etcd arguments 2021-01-29 19:43:48 -06:00
Travis
1c0b926eae
Merge master into disco 2021-01-28 18:03:38 -06:00
Travis
4380a05bbd
address some coord/node0 test issues 2021-01-26 22:46:37 -06:00
Travis
ace4dea46f
address some test failures due to random ordered etcd ID 2021-01-25 00:52:49 -06:00
Travis
1d55e671a2
go mod tidy and linter 
fix race

cleanup
 2021-01-15 17:48:44 -06:00
Travis
9855f4d0a0
Merge branch 'travis-test-ci' into disco-try 2021-01-15 14:31:19 -06:00
Travis
10380a1da1
Implement snap := ClusterSnapshot() 
Below is the list of instance of `ClusterSnapshot()` in the latest
`with-etcd` code. Some of these may not yet exist in the `disco` branch,
but this commit is implementing any that currently apply.

==========================
Done:
==========================
index.go
930:	snap := NewClusterSnapshot(topo, topo.Hasher, topo.ReplicaN)
1072:	snap := NewClusterSnapshot(topo, topo.Hasher, topo.ReplicaN)

cmd/pilosa-fsck/fsck.go
786:	snap := pilosa.NewClusterSnapshot(cfg.topo, cfg.topo.Hasher, cfg.topo.ReplicaN)

boltdb/translate.go
558:	snap := pilosa.NewClusterSnapshot(topo, topo.Hasher, topo.ReplicaN)
1264:	snap := pilosa.NewClusterSnapshot(topo, topo.Hasher, topo.ReplicaN)

fragment.go
3448:	snap := NewClusterSnapshot(s.Cluster.noder, s.Cluster.Hasher, s.Cluster.ReplicaN)
3568:	snap := NewClusterSnapshot(s.Cluster.noder, s.Cluster.Hasher, s.Cluster.ReplicaN)
3620:	snap := NewClusterSnapshot(s.Cluster.noder, s.Cluster.Hasher, s.Cluster.ReplicaN)

==========================
Remaining:
==========================

cluster.go
371:	snap := NewClusterSnapshot(NewLocalNoder(nodes), c.Hasher, c.ReplicaN)
474:	snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
639:	fSnap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
640:	toSnap := NewClusterSnapshot(to.noder, c.Hasher, to.ReplicaN)
703:	snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
1475:		snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
1502:	snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
1941:	snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
1986:	snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
2049:	snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
2126:	snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)

api.go
475:	snap := NewClusterSnapshot(api.cluster.noder, api.cluster.Hasher, api.cluster.ReplicaN)
604:	snap := NewClusterSnapshot(api.cluster.noder, api.cluster.Hasher, api.cluster.ReplicaN)
690:	snap := NewClusterSnapshot(api.cluster.noder, api.cluster.Hasher, api.cluster.ReplicaN)
1684:	snap := NewClusterSnapshot(api.cluster.noder, api.cluster.Hasher, api.cluster.ReplicaN)
1946:	snap := NewClusterSnapshot(api.cluster.noder, api.cluster.Hasher, api.cluster.ReplicaN)

executor.go
3781:	snap := NewClusterSnapshot(e.Cluster.noder, e.Cluster.Hasher, e.Cluster.ReplicaN)
4157:	snap := NewClusterSnapshot(e.Cluster.noder, e.Cluster.Hasher, e.Cluster.ReplicaN)
4200:	snap := NewClusterSnapshot(e.Cluster.noder, e.Cluster.Hasher, e.Cluster.ReplicaN)
4243:	snap := NewClusterSnapshot(e.Cluster.noder, e.Cluster.Hasher, e.Cluster.ReplicaN)
4517:	snap := NewClusterSnapshot(NewLocalNoder(e.Cluster.Nodes()), e.Cluster.Hasher, e.Cluster.ReplicaN)

holder.go
1465:	snap := NewClusterSnapshot(s.Cluster.noder, s.Cluster.Hasher, s.Cluster.ReplicaN)
1668:	snap := NewClusterSnapshot(s.Cluster.noder, s.Cluster.Hasher, s.Cluster.ReplicaN)
1889:	snap := NewClusterSnapshot(s.Cluster.noder, s.Cluster.Hasher, s.Cluster.ReplicaN)
1963:	snap := NewClusterSnapshot(c.Cluster.noder, c.Cluster.Hasher, c.Cluster.ReplicaN)
 2021-01-15 11:47:37 -06:00
Travis
7875fb8e5a
remove pilosa.DefaultPartitionN 2021-01-15 11:47:37 -06:00
Travis
ff2d235702
change all references to use subpackages: topology, net 2021-01-15 11:47:36 -06:00
Travis
27614c42f7
Finish implementing port wrapper 2021-01-13 22:56:54 -06:00
Travis
bc13834343
disco/etcd work: fix lots of races, start all cluster nodes at once. 
port mapper gives out ports from 63000-65000 for the tests

fix another race

http test uses port.MustGetPort

rbf: remove :0 port request

ocd happy

test fix for grpc listener address already in use

test/disco allocates BindGRPC port from the port mapper

dump stack on each GetPort

verify each port is usable right away

server/config.go has Config.Validate() now

panic if gossip port is 0. validate server.Config

fix another gossip port 0

builds

quiet, don't dump stack on each port alloc

builds

happy linter

even gossip fallback should not be zero but rather use the port mapper
 2021-01-12 21:06:12 -06:00
Ben Johnson
b7973e2612 Add read benchmarks 2021-01-12 13:48:09 -07:00
Travis
134abda51b
Implement snap := ClusterSnapshot() 
Below is the list of instance of `ClusterSnapshot()` in the latest
`with-etcd` code. Some of these may not yet exist in the `disco` branch,
but this commit is implementing any that currently apply.

==========================
Done:
==========================
index.go
930:	snap := NewClusterSnapshot(topo, topo.Hasher, topo.ReplicaN)
1072:	snap := NewClusterSnapshot(topo, topo.Hasher, topo.ReplicaN)

cmd/pilosa-fsck/fsck.go
786:	snap := pilosa.NewClusterSnapshot(cfg.topo, cfg.topo.Hasher, cfg.topo.ReplicaN)

boltdb/translate.go
558:	snap := pilosa.NewClusterSnapshot(topo, topo.Hasher, topo.ReplicaN)
1264:	snap := pilosa.NewClusterSnapshot(topo, topo.Hasher, topo.ReplicaN)

fragment.go
3448:	snap := NewClusterSnapshot(s.Cluster.noder, s.Cluster.Hasher, s.Cluster.ReplicaN)
3568:	snap := NewClusterSnapshot(s.Cluster.noder, s.Cluster.Hasher, s.Cluster.ReplicaN)
3620:	snap := NewClusterSnapshot(s.Cluster.noder, s.Cluster.Hasher, s.Cluster.ReplicaN)

==========================
Remaining:
==========================

cluster.go
371:	snap := NewClusterSnapshot(NewLocalNoder(nodes), c.Hasher, c.ReplicaN)
474:	snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
639:	fSnap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
640:	toSnap := NewClusterSnapshot(to.noder, c.Hasher, to.ReplicaN)
703:	snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
1475:		snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
1502:	snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
1941:	snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
1986:	snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
2049:	snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)
2126:	snap := NewClusterSnapshot(c.noder, c.Hasher, c.ReplicaN)

api.go
475:	snap := NewClusterSnapshot(api.cluster.noder, api.cluster.Hasher, api.cluster.ReplicaN)
604:	snap := NewClusterSnapshot(api.cluster.noder, api.cluster.Hasher, api.cluster.ReplicaN)
690:	snap := NewClusterSnapshot(api.cluster.noder, api.cluster.Hasher, api.cluster.ReplicaN)
1684:	snap := NewClusterSnapshot(api.cluster.noder, api.cluster.Hasher, api.cluster.ReplicaN)
1946:	snap := NewClusterSnapshot(api.cluster.noder, api.cluster.Hasher, api.cluster.ReplicaN)

executor.go
3781:	snap := NewClusterSnapshot(e.Cluster.noder, e.Cluster.Hasher, e.Cluster.ReplicaN)
4157:	snap := NewClusterSnapshot(e.Cluster.noder, e.Cluster.Hasher, e.Cluster.ReplicaN)
4200:	snap := NewClusterSnapshot(e.Cluster.noder, e.Cluster.Hasher, e.Cluster.ReplicaN)
4243:	snap := NewClusterSnapshot(e.Cluster.noder, e.Cluster.Hasher, e.Cluster.ReplicaN)
4517:	snap := NewClusterSnapshot(NewLocalNoder(e.Cluster.Nodes()), e.Cluster.Hasher, e.Cluster.ReplicaN)

holder.go
1465:	snap := NewClusterSnapshot(s.Cluster.noder, s.Cluster.Hasher, s.Cluster.ReplicaN)
1668:	snap := NewClusterSnapshot(s.Cluster.noder, s.Cluster.Hasher, s.Cluster.ReplicaN)
1889:	snap := NewClusterSnapshot(s.Cluster.noder, s.Cluster.Hasher, s.Cluster.ReplicaN)
1963:	snap := NewClusterSnapshot(c.Cluster.noder, c.Cluster.Hasher, c.Cluster.ReplicaN)
 2021-01-06 22:53:20 -06:00
Travis
3f26d667b4
remove pilosa.DefaultPartitionN 2021-01-06 22:45:44 -06:00
Travis
4515a24e48
change all references to use subpackages: topology, net 2021-01-06 16:09:24 -06:00
Matt Jaffee
bed2cffd5e
fix test failures in case of running Pilosa on system 
If you're running a Pilosa with mostly default configuration on your
system, some of these tests would fail due to things like port
conflicts. These changes address the most common failures.
 2020-12-29 13:57:18 -06:00
Maxton Huff
dee700741a deprecate cluster.long-query-time and create long-query-time 
moved lonquerytime from cluster into server and moved cluster.longquerytime into top level config
kept cluster.longquerytime for backwards compatibility, favored if both longquerytime options are present
 2020-12-10 10:51:31 -06:00
Cody Soyland
bc94c7bcdf Remove lmdb dependency and references, vendor Barrier 2020-12-09 08:43:23 -06:00
Seebs
3d802f428b add Distinct, be more consistent about using correct rand.Rand 
The RandomQueryConfig has its own seeded RNG, but we didn't always use
it (especially in the last two commits, but also I think in one previous
thing), so let's use it more consistently.

We check for int fields (as opposed to decimal), and if we find them,
we add Distinct to our list of potential queries to use if and only
if we've got a depth of at least one so there'd be a child query under
the current query, and if we do, grab one of the int (not decimal)
fields and do a query on that.
 2020-11-23 09:54:59 -06:00
Seebs
796d646a0c handle integer fields 
Extend field handling to include int (and decimal) fields, allowing them
to have range operations specified on them.
 2020-11-23 09:51:29 -06:00
Seebs
2a2a8abbd4 support time fields 
For time fields, allow specifying a range of times, then 19/20 times,
specify "from" and "to" times in that range when querying those fields,
rather than just looking at the standard view all the time.
 2020-11-23 09:51:29 -06:00
Seebs
48b97d6ad4 report QPS after last query 
More generally, report QPS not at 0 queries, which is boring, but every
100 queries *and* after the last query if the last query wasn't at a
multiple of 100 queries. Makes the output slightly more useful, I think.
 2020-11-23 09:51:29 -06:00
Ben Johnson
8de1959938 Remove RBF exclusive/direct write. 2020-11-09 08:23:09 -07:00
Jason E. Aten
458095a707 rbf default. Add TODO comments, slurp -profile returns a cpu profile 
- default Tx is once again RBF, changed from bolt.
- document the RBF code review comments that were not addressed
  before #1052 was merged, so they don't get lost.
- they should be easily addressed by replaying the entire WAL file
  rather than from the DB meta page 0 notion of the last WalID
- cleanup rbf/cfg/cfg.go stale comments, ensure default0 respected.
  1 msec checkpoint time, 1MB wal segment defaults.
- return a specific error, ErrNoMetaFound, from findNextWALMetaPage()
  rather than io.EOF, since there actually wasn't any file IO involved.
- add http handlers for /cpu-profile/start and /cpu-profile/stop
  in http/handler.go enable CPU profiling at specific time points
  during an ingest or other operation.
 2020-11-04 01:36:17 +00:00
Ben Johnson
d5369a084e Improve RBF CLI help 2020-10-30 07:25:36 -06:00
Ben Johnson
2b1d790625 Fix tree key formatting; add --with-tree option 2020-10-27 08:24:56 -06:00
Ben Johnson
7fa1a5edd2 Add RBF CLI commands 2020-10-27 08:12:22 -06:00
jaten-molecula
20d4268676
Merge branch 'master' into rbf_config 2020-10-14 07:19:37 -05:00
Jason E. Aten
50a8db854d pilosa-fsck: the -readers flag controls parallelism 
- add path info to the panic if we find a corrupt boltdb
   translation store.
 2020-10-13 18:58:38 -05:00
J
3ad8ec7f0f rbf: add runtime options to DB struct 
- FsyncEnabled and DoAllocZero moved to DB struct.
 - deletes unused xrbrsupport.go and cmd/convert
 - fixes #941
 2020-10-12 18:35:00 -05:00