Commit graph

8920 commits

Author SHA1 Message Date
Alan Bernstein
474261f12f Add single-node 'bytesOnDisk' object to /status response 2020-10-01 10:25:48 -05:00
Nia
cac89f4c96
Merge pull request #913 from niaow/misaligned-distinct
Fix misaligned bitmaps in Distinct
2020-10-01 11:17:29 -04:00
Nia Weiss
e0f9870378
fix misaligned bitmaps in Distinct
This resolves an issue in which a filter would eliminate all bits in all shards other than shard 0.
2020-10-01 10:07:08 -04:00
Nia
f030792b8a
Merge pull request #905 from niaow/syncrace
Fix race condition when resetting translation
2020-09-30 09:26:43 -04:00
Nia
c31d2f143c
Merge branch 'master' into syncrace 2020-09-30 09:17:59 -04:00
Cody Soyland
86c5e040b1
Merge pull request #870 from codysoyland/grpc-compat
Fully backwards-compatible VDSM gRPC interface
2020-09-29 15:58:20 -06:00
Jason Aten
0e8564e2be fix license exceptions for vdsm/proto 2020-09-29 15:30:41 -05:00
Cody Soyland
949ca882fc Remove unused helper func 2020-09-29 15:30:12 -05:00
Cody Soyland
fdce92eb40 Use shared types from pilosa proto file and enable generic proxying of query calls 2020-09-29 15:30:12 -05:00
Cody Soyland
83704e7973 Proxy unary methods to Pilosa gRPC handler 2020-09-29 15:29:37 -05:00
Cody Soyland
c1e50ed909 Remove VDS references from pilosa proto file, begin porting VDSM service 2020-09-29 15:29:37 -05:00
Nia Weiss
1b210080c6
fix race condition when resetting translation
Previously, we never waited for translation sync goroutines to stop.
That issue should be mostly harmless in the normal path.
Additionally, this waits for the translation sync to shut down when stopping the server.
2020-09-29 13:54:16 -04:00
Seebs
fbb648b1fb enforce append-like semantics for *Container more consistently
The copy-on-write/rowCache changes require that functions that
modify containers be able to generate new containers. Once that
became possible, some significant pool of other operations
started relying on it -- for instance, operations might return
a new container even though they're in theory "in place" operations.

I developed a tool for checking for unused function return
values (github.com/molecula/noticeme), and ran it on this, and
picked out the places where `*Container` values were generated
but not used, and some of them seem to be potentially-real
bugs, and a few are probably harmless. Updated code to make
those diagnostics go away.
2020-09-28 12:19:24 -05:00
tgruben
82cdffd309
Merge pull request #899 from jaten-molecula/better_err_on_closed_db
better error reporting when making Tx on closed lmdb
2020-09-26 09:59:15 -05:00
Jason Aten
5a3170a9a4 better error reporting when making Tx on closed lmdb
- return error instead of panic.
2020-09-26 06:24:58 -05:00
Nia
3d07d13a7c
Merge pull request #897 from niaow/no-explicit-recalculate
Stop explicitly recalculating caches in tests
2020-09-25 11:18:37 -04:00
Nia Weiss
79f68f7a2e
stop explicitly recalculating caches in tests
This is no longer necessary, as caches now recalculate on read.
Also, in general a user will not explicitly request recalculation, so it would make sense for our tests to reflect that.
2020-09-25 10:36:00 -04:00
jaten-molecula
f026c43649
Merge pull request #891 from molecula/optimize_directadd
Bitmap.DirectAdd avoids returning overfull containers
2020-09-23 17:25:29 -05:00
Jason Aten
7ae8accfa8 Bitmap.DirectAdd avoids returning overfull containers
- log Debugf when we repair a fragment block
- better run-run roaring testing for over-sized containers
- add which fragment path to panic on container too big
- include container contents in roaring hash for pilosa-chk/pilosa-check-backup
2020-09-23 17:13:25 -05:00
Nia
4b5572da1d
Merge pull request #893 from niaow/limit-cluster
Fix Limit when used as an input to another query
2020-09-23 13:51:11 -04:00
Nia
e0f25f1d20
Merge branch 'master' into limit-cluster 2020-09-23 13:46:02 -04:00
Nia Weiss
2158417f34
fix Limit when used as an input to another query 2020-09-23 13:08:48 -04:00
Kuba Podgórski
8b855b97d0
Merge pull request #892 from kuba--/fix-545
Translate field IDs on coordinator
2020-09-23 17:49:15 +02:00
Kuba Podgórski
ba5438e02b Trabnslate field IDs on coordinator 2020-09-23 15:21:33 +02:00
jaten-molecula
f7afd5d2e1
Merge pull request #875 from molecula/remove_debug_tooling
turn on row cache. tx: remove debug tooling
2020-09-22 15:56:02 -05:00
Jason Aten
55d4c29933 turn off debug machinery on tx backends
- enable row cache again. Was off for tx perf measurement.
 - centralize UseRowCache choice to just one point, in rbf.EnableRowCache
2020-09-22 15:47:52 -05:00
Kuba Podgórski
886318931d
Merge pull request #882 from kuba--/fix-869
Fix holes in grpc response for inspect
2020-09-22 14:20:10 +02:00
Kuba Podgórski
b54a289cd8 Fix holes in grpc response for inspect 2020-09-22 12:02:26 +02:00
jaten-molecula
ba2851229d
Merge pull request #871 from molecula/bluegreen_verify
blue_green verify accepts empty fragments
2020-09-18 11:55:22 -04:00
Jason Aten
57be5392cf blue_green verify accepts empty fragments 2020-09-18 10:49:36 -05:00
tgruben
8ec10ec1e8
Merge pull request #868 from molecula/with_primary_instead_owner
Translate only on coordinator/primary
2020-09-17 17:12:56 -05:00
Kuba Podgórski
650244214d Translate only on coordinator/primary
- This is the commit message #3:
2020-09-17 17:03:29 -05:00
jaten-molecula
7dc59c4d9c
Merge pull request #831 from molecula/rbf-tx-cursor-stack-allocate
Stack allocate return from rbf.Tx.cursor()
2020-09-16 23:40:32 -04:00
Jason Aten
9b99009871 allow rbf-tx-cursor-stack-allocate to merge; fix conflict with the new HasData code 2020-09-16 22:35:30 -05:00
jaten-molecula
c04642f34b
Merge branch 'master' into rbf-tx-cursor-stack-allocate 2020-09-16 23:21:35 -04:00
jaten-molecula
0f5838816b
Merge pull request #857 from molecula/rbf-fix-direct-write-corruption
Fix RBF write corruption during direct write.
2020-09-16 23:07:57 -04:00
jaten-molecula
71033f78b5
Merge branch 'master' into rbf-fix-direct-write-corruption 2020-09-16 22:32:47 -04:00
Nia
16c5212e2d
Merge pull request #858 from niaow/deprecate-inspect
Add a deprecation warning to Inspect
2020-09-16 15:31:57 -04:00
Nia Weiss
f03ba0e682
add a deprecation warning to Inspect 2020-09-16 12:30:12 -04:00
Ben Johnson
6918fe2f60
Merge branch 'master' into rbf-fix-direct-write-corruption 2020-09-16 09:55:26 -06:00
jaten-molecula
75b01ba87d
Merge pull request #850 from molecula/db_has_data_rb
blue_green verification and migration capabilities.
2020-09-16 11:22:34 -04:00
Ben Johnson
458984c756 Fix RBF write corruption during direct write.
This commit fixes an issue where direct writes would overwrite the
source page where data was being copied from because writes are
immediate (instead of going to the WAL first).
2020-09-16 08:59:53 -06:00
Jason Aten
31d54010f8 blue_green verification and migration capabilities.
- on startup in blue_green mode, we will migrate
   blue to green if blue is empty.
 - otherwise, when blue has data, we verify
   against green before proceeding with the
   blue_green run.
 - small optimization in the rbf cursorx.go to
   short-circuit processing on a nil bitmap.
   This avoids a roaringparanoia tag panic.
 - back out holdbkg.go, was too slow.
   add a distinct Holder.imu lock instead.
2020-09-16 09:41:05 -05:00
Kuba Podgórski
dd4ffff704
Merge pull request #853 from kuba--/fix-translate_index_keys
Fix translation index keys
2020-09-16 15:21:31 +02:00
Kuba Podgórski
fa159c9a17
Merge branch 'master' into fix-translate_index_keys 2020-09-16 15:09:49 +02:00
jaten-molecula
47926092ee
Merge pull request #855 from molecula/cleanup_cherrypick
cleanup lmdb tests that were leaving test directories behind
2020-09-16 07:51:23 -04:00
Jason Aten
c093aa9d0e cleanup lmdb tests 2020-09-16 06:46:12 -05:00
Kuba Podgórski
ab26134992
Merge branch 'master' into fix-translate_index_keys 2020-09-16 10:39:32 +02:00
Kuba Podgórski
a8a9a73b6c Fix translation index keys 2020-09-16 10:36:44 +02:00
tgruben
793cf934b8
Merge pull request #848 from molecula/fix_843
don't panic if dbs is reopened under roaring only. fixes #843
2020-09-15 16:54:52 -05:00