scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-10-09 20:37:52 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
8305 commits 45 branches 174 tags 85 MiB
Commit graph

8305 commits

This branch
This branch
All branches
Author SHA1 Message Date
Alan Bernstein
6a298590be Set omitempty for disk capacity json 2020-10-14 12:20:05 -05:00
Alan Bernstein
ba4ca1a93e Include disk capacity in usage response 2020-10-14 12:20:05 -05:00
Alan Bernstein
2a44bdd25c Collect size-on-disk usage data from all nodes 2020-10-14 12:20:05 -05:00
Cody Soyland
e1e0b298f5
Merge pull request #964 from codysoyland/check-changelog-label 
Check for changelog label in CI
 2020-10-14 10:11:45 -05:00
Seebs
c88192b4ac Check more carefully for, and also fix, containers with invalid N 
In rare cases, RBF can produce containers which have a recorded N value which
is incorrect. This rarely affects anything, but on some particular queries,
this can result in very strange outcomes, like array containers with more
than 1<<16 entries.

To fix this, we have toContainer specify that it doesn't know the correct
N for the bitmap containers it's creating, which costs extra time for counting,
and should be considered a temporary workaround.

Also, we add a CheckN() function which is controlled by the
roaringparanoia flag, and add a number of calls to it, for instance, as
deferred calls after every container operation when roaringparanoia is
enabled. This means that we get improved confidence that we've caught
the relevant errors, but is not suitable for production use.
 2020-10-14 10:03:18 -05:00
Cody Soyland
ca5dbe8d39 Fix pull request filter 2020-10-14 09:42:07 -05:00
Cody Soyland
2ac09af894 Only run on pull requests 2020-10-14 09:29:51 -05:00
Cody Soyland
4e66bbb76b Check for changelog label in CI 2020-10-14 09:29:51 -05:00
seebs
6c14aa965d
Merge pull request #895 from seebs/appendSemantics 
use append-like semantics consistently for *Container
 2020-10-14 09:04:56 -05:00
jaten-molecula
3d4da8f51b
Merge branch 'master' into appendSemantics 2020-10-14 07:24:18 -05:00
jaten-molecula
20d4268676
Merge branch 'master' into rbf_config 2020-10-14 07:19:37 -05:00
jaten-molecula
a997e713dc
Merge pull request #977 from molecula/readers 
pilosa-fsck: the -readers flag controls parallelism
 2020-10-13 19:14:09 -05:00
Jason E. Aten
50a8db854d pilosa-fsck: the -readers flag controls parallelism 
- add path info to the panic if we find a corrupt boltdb
   translation store.
 2020-10-13 18:58:38 -05:00
J
3ad8ec7f0f rbf: add runtime options to DB struct 
- FsyncEnabled and DoAllocZero moved to DB struct.
 - deletes unused xrbrsupport.go and cmd/convert
 - fixes #941
 2020-10-12 18:35:00 -05:00
tgruben
db9e8cd8ef
Merge pull request #966 from molecula/fix843 
pilosa/dbshard: allow deleteIndex and then re-use of index
 2020-10-12 18:10:04 -05:00
Jason E. Aten
b3e6cdc0d8 pilosa/dbshard: allow deleteIndex and then re-use of index 
- fixes #843
 2020-10-12 17:46:06 -05:00
jaten-molecula
54255eae15
Merge pull request #957 from molecula/fix-tx-deadlock 
Fix deadlock in importWorker()
 2020-10-12 17:04:11 -05:00
Ben Johnson
676208ea73 Add test to reproduce deadlock 2020-10-12 12:32:58 -06:00
Ben Johnson
9b83b71c8f Fix error check reference in importWorker() 2020-10-12 11:28:18 -06:00
Ben Johnson
bfe570ef9d Fix deadlock in importWorker() 
This commit wraps the work for a view in a function so that the
defer on the finisher executes after each view is processed instead
of at the end.
 2020-10-12 11:25:28 -06:00
Cody Soyland
279be13a46
Merge pull request #958 from codysoyland/grpc-compat-update 
Better VDSM gRPC compatibility
 2020-10-12 09:27:04 -05:00
Cody Soyland
e40b224a4f
Merge branch 'master' into grpc-compat-update 2020-10-12 09:08:42 -05:00
tgruben
d3811634d6
Merge pull request #959 from molecula/parallel_fsck 
pilosa-fsck: parallelize Index.ComputeTranslatorSummary
 2020-10-09 16:12:36 -05:00
Jason Aten
8da15b18d4 pilosa-fsck: parallelize Index.ComputeTranslatorSummary 
- add -index option
 2020-10-09 15:53:29 -05:00
Cody Soyland
6c10917ceb Use vdsm.QueryPQLRequest for vdsm.QueryPQLUnary 2020-10-09 13:34:20 -05:00
Cody Soyland
3039aa44af Use copy of vdsm InspectRequest for better compatibility 2020-10-09 13:22:24 -05:00
Cody Soyland
a58c345232 Use copy of vdsm QueryPQLRequest for better compatibility 2020-10-09 13:22:24 -05:00
tgruben
d3ce4fa70e
Merge pull request #948 from molecula/randomquery 
pilosa/cmd/random-query: generate random queries from existing schema/data
 2020-10-09 12:16:28 -05:00
Jason Aten
937a1afade pilosa/cmd/random-query: generate random queries from existing schema 
- fixes molecula/molecula#136
 - flag -n 0 does continuous queries until process is killed.
 2020-10-08 18:38:01 -05:00
jaten-molecula
06ee8b16ac
Merge pull request #940 from molecula/fsck_2index_repair 
pilosa-fsck: multi-index counts and repairs.
 2020-10-07 07:35:08 -05:00
Jason Aten
a63c6ee2b2 pilosa-fsck: multi-index counts and repairs. 
- multiple indexes repaired at once could crosstalk. Fixed.
 - the counts of keys and ids are now broken down by index.
 2020-10-06 21:10:52 -05:00
tgruben
d41de17503
Merge pull request #931 from molecula/fsck_msg 
pilosa-fsck: correct repair -> analysis in pilosa-fsck logging
 2020-10-06 13:21:21 -05:00
jaten-molecula
bc8fc23ac9
Merge branch 'master' into fsck_msg 2020-10-06 13:12:46 -05:00
tgruben
3a742a0caa
Merge pull request #938 from molecula/ci_rbf 
run CI on rbf and rbf_lmdb
 2020-10-06 12:52:33 -05:00
Jason Aten
83f230bbfb run CI on rbf and rbf_lmdb 2020-10-06 12:22:03 -05:00
Ben Johnson
ab6581446d
Merge pull request #889 from molecula/rbf-immutable-wal 2020-10-06 10:45:47 -06:00
Ben Johnson
3b7758a4f2 Add comment explaining rbf.DB.checkpoint() args 2020-10-06 10:36:57 -06:00
Ben Johnson
2440413c49 Remove test skips based on race detector 2020-10-06 10:29:17 -06:00
Ben Johnson
7b64abbb53 Reduce default RBF DB size to 4GB; remove race skips 2020-10-06 09:36:31 -06:00
Ben Johnson
644969e6a9 Add log.* and tourna.* to gitignore 2020-10-06 09:36:31 -06:00
Ben Johnson
265452cf3d Add rbf.SyncEnabled 2020-10-06 09:36:31 -06:00
Ben Johnson
cf208cfaa3 Increase test http client timeout 
The timeout was increased to allow additional time for RBF to process
a lot of individual `Set()` commands in `TestMain_RecalculateHashes`.
 2020-10-06 09:36:31 -06:00
Ben Johnson
3852ac79c4 Add rbf.DB.TxN() function and test check 2020-10-06 09:36:31 -06:00
Ben Johnson
c978e2242e Skip some RBF tests during race detection 2020-10-06 09:36:31 -06:00
Ben Johnson
c51ba69c09 Fix RBF sync calls 2020-10-06 09:36:31 -06:00
Ben Johnson
3429421148 Fix RBF checkpoint off-by-one WAL ID issue 2020-10-06 09:36:31 -06:00
Ben Johnson
d826cec4b3 fix RBF WAL segment reference error 2020-10-06 09:36:31 -06:00
Ben Johnson
cdd10a26f0 Refactor RBF to use immutable list of WAL segments. 
Previously, RBF shared a list of WAL segments between the DB & Tx.
However, this increased the need for mutexes to access the data.

WAL segments are effectively immutable on-disk so the list of segments
has been refactored so that changes to the segment list are done via
copy-on-write which allows read transactions to access segment data
without a mutex.

The database checkpointing can remove early, unused segments and
there is an update/add check to make sure that Tx segments pushed back
to the DB do not include removed segments.
 2020-10-06 09:36:31 -06:00
tgruben
8dbfaf5045
Merge pull request #932 from jaten-molecula/pr919 
api usage should ignore tx stores. fixes #919
 2020-10-06 09:45:43 -05:00
jaten-molecula
ce3a51eb87
Merge branch 'master' into fsck_msg 2020-10-06 09:28:36 -05:00