scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-09-05 08:10:50 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
6415 commits 45 branches 174 tags 85 MiB
Commit graph

6415 commits

This branch
This branch
All branches
Author SHA1 Message Date
jaten-molecula
cb73c71e02
Merge pull request #1072 from molecula/uni-wal 
Refactor RBF to use a single WAL file
 2020-11-11 15:44:45 -06:00
jaten-molecula
563c70dc43
Merge branch 'master' into uni-wal 2020-11-11 15:27:15 -06:00
tgruben
b3d2704066
Merge pull request #1085 from jaten-molecula/uniq_dbnames 
unique db names so parallel test runs don't collide
 2020-11-11 14:26:04 -06:00
Jason E. Aten
fd9210b31c unique db names so parallel test runs don't collide 2020-11-11 20:04:02 +00:00
Ben Johnson
81a64c5902 Add RBF halting; remove time based checkpoint 2020-11-11 11:17:07 -07:00
Ben Johnson
9eba299d35 Restrict max RBF transaction size 2020-11-10 08:14:14 -07:00
Ben Johnson
78eb9e0711 Fix linter 2020-11-10 07:07:06 -07:00
Ben Johnson
52340212f6 Add RBF dirty page cache 2020-11-09 16:11:20 -07:00
Ben Johnson
8de1959938 Remove RBF exclusive/direct write. 2020-11-09 08:23:09 -07:00
Ben Johnson
3554048877 Refactor RBF to use a single WAL file 2020-11-09 08:03:22 -07:00
tgruben
e99f7f0fc3
Merge pull request #1074 from jaten-molecula/fsync_applies_to_all 
apply --fsync flag to all tx backend
 2020-11-06 12:34:28 -06:00
Jason E. Aten
a38d4fa46c apply --fsync flag to all tx backend 
- rename from --rbf-fsync to --fsync, as it
   now effects bolt, lmdb too.
 2020-11-06 18:11:02 +00:00
jaten-molecula
822c7482a5
Merge pull request #1071 from jaten-molecula/parallel_migration_rb 
run migration in parallel
 2020-11-05 14:09:31 -06:00
Jason E. Aten
7e22994a6d run migration in parallel 
- migration can be slow. parallelize it.
 2020-11-05 18:31:27 +00:00
tgruben
bc26de63b0
Merge pull request #1070 from jaten-molecula/slurp_profile_rbf_comments 
rbf default. Add TODO comments, slurp -profile returns a cpu profile
 2020-11-03 19:52:54 -06:00
Jason E. Aten
458095a707 rbf default. Add TODO comments, slurp -profile returns a cpu profile 
- default Tx is once again RBF, changed from bolt.
- document the RBF code review comments that were not addressed
  before #1052 was merged, so they don't get lost.
- they should be easily addressed by replaying the entire WAL file
  rather than from the DB meta page 0 notion of the last WalID
- cleanup rbf/cfg/cfg.go stale comments, ensure default0 respected.
  1 msec checkpoint time, 1MB wal segment defaults.
- return a specific error, ErrNoMetaFound, from findNextWALMetaPage()
  rather than io.EOF, since there actually wasn't any file IO involved.
- add http handlers for /cpu-profile/start and /cpu-profile/stop
  in http/handler.go enable CPU profiling at specific time points
  during an ingest or other operation.
 2020-11-04 01:36:17 +00:00
tgruben
a55cdad139
Merge pull request #1069 from jaten-molecula/faster_new_tx 
take a mutex off the fast path of TxFactory.NewTx
 2020-11-03 19:15:26 -06:00
Jason E. Aten
bf36f54b8d take a mutex off the fast path of TxFactory.NewTx 2020-11-04 01:04:04 +00:00
tgruben
40a773b839
Merge pull request #1068 from jaten-molecula/qcx_gettx_error 
qcx.GetTx returns an error
 2020-11-03 18:34:24 -06:00
Jason E. Aten
6aadd13095 qcx.GetTx returns an error 
- to indicate that the query context is already
   done.
 - handles the case where the import worker is
   interrupted early by a ctx cancellation,
   thus avoiding a panic.
 2020-11-04 00:01:02 +00:00
Ben Johnson
06cb04f87a
Merge pull request #1052 from molecula/fix-wal-not-found 
Fix RBF WAL ID panic.
 2020-11-03 11:16:33 -07:00
Ben Johnson
731a1ef25e Refactor RBF WAL to only only checkpoint-in-full. 2020-11-03 10:41:31 -07:00
Ben Johnson
ea3732fa62 Fix WAL ID not found panic. 
This commit changes the checkpointing to determine a minimum WAL ID
for readers and a max ID based on the writer. Pages are checkpointed
from the WAL up to the writer's max WAL ID but segments are removed
only up to the reader's minimum WAL ID. This ensures that WAL pages
are not removed out from under current read transactions.
 2020-11-03 10:41:31 -07:00
Ben Johnson
c47b49a06e
Merge pull request #1053 from molecula/increase-rbf-wal-write-cache 2020-11-03 07:37:05 -07:00
Ben Johnson
db74f0dffa Increase RBF WAL write cache size to 1MB. 
Previously we dropped the cache size to 64KB but that seems much
too low. Write performance suffers considerably.
 2020-10-30 17:09:33 -06:00
Matthew Jaffee
3ff85ccb53
Merge pull request #1051 from molecula/default-txn-bolt-fix-ci 
default to Bolt txn due to CI failures blocking other team
 2020-10-30 15:01:57 -05:00
Matt Jaffee
83dd8026d5
default to Bolt txn due to CI failures blocking other team 
Our nightly CI has been failing for a week due to WAL issues and it's
making it difficult for Kuba and Antonio to do things on the
integration repo. Hoping bolt backend will solve that in the short
term. I think the issue is Pilosa #1046 (that's from memory though)
 2020-10-30 13:55:36 -05:00
Ben Johnson
bb5153a7c1
Merge pull request #1049 from molecula/rbf-cli-help 
Improve RBF CLI help
 2020-10-30 08:38:14 -06:00
Ben Johnson
d5369a084e Improve RBF CLI help 2020-10-30 07:25:36 -06:00
tgruben
f446eee088
Merge pull request #1048 from jaten-molecula/delete_empty 
DeleteEmptyContainer true by default now
 2020-10-30 06:30:09 -05:00
Jason E. Aten
9c66cd5a81 DeleteEmptyContainer true by default now 2020-10-30 04:15:53 +00:00
tgruben
4113a3d85b
Merge pull request #933 from molecula/qcx_no_auto_reset 
pilosa: no automatic reuse of Qcx
 2020-10-29 20:03:42 -05:00
Jason Aten
016e5e0774 no automatic reuse of Qcx 
Per slack discussion with Seebs and Nia,
we'll try not automatically resetting the Qcx.

The worry was that our goroutine shutdown
management is so poor that we are asking for
GetTx on a goroutine that still has a Qcx
from a query that was cancelled.

If this is the case, we will now panic instead of
issuing a new Tx. Then we can fix the poor
goroutine management.

 - also require Qcx.Finish or Abort before Reset
 2020-10-30 00:36:46 +00:00
tgruben
51c9f4feeb
Merge pull request #1044 from jaten-molecula/startup_shards 
don't apply startup shard cache to roaring with a specified view path
 2020-10-29 18:24:57 -05:00
Jason E. Aten
18e253870e cleanup TypedDBPerShardGetShardsForIndex logic 2020-10-29 22:25:06 +00:00
Jason E. Aten
f8e7b27a6f don't apply startup shard cache to roaring with a specified view path 
- avoids creating a new empty 8 byte shard file under all the
   views that don't have them already.
 2020-10-29 22:09:55 +00:00
tgruben
ed6f59a827
Merge pull request #1042 from jaten-molecula/migration_logging 
better migration logging
 2020-10-29 04:38:53 -05:00
Jason E. Aten
2d55ffbd28 short circuit if no data to migrate 2020-10-29 02:44:22 +00:00
Jason E. Aten
d4df15721d allow migration of empty to empty 2020-10-29 02:41:09 +00:00
Jason E. Aten
aa47cf5bf4 fix verify test 2020-10-29 02:35:38 +00:00
Jason E. Aten
0e354aedf2 error expected trying to migration to roaring 2020-10-29 02:29:57 +00:00
Jason E. Aten
f9f94aab0d fix typo 2020-10-29 02:15:43 +00:00
Jason E. Aten
bceabc9127 error on un-implemented migrate to roaring 2020-10-29 02:11:28 +00:00
Jason E. Aten
3baa8ea0f6 check if roaring data present with RoaringHasData 2020-10-29 01:47:26 +00:00
Jason E. Aten
639e9b3bf0 quiet 2020-10-29 01:32:04 +00:00
Jason E. Aten
6e20e6439f more logging 2020-10-29 01:30:16 +00:00
Jason E. Aten
b1ff3791b8 better error when green does not exist on migration 2020-10-29 01:20:41 +00:00
Jason E. Aten
85fc785296 timed progress 2020-10-29 01:12:19 +00:00
Jason E. Aten
509348260f better migration logging 2020-10-28 22:54:58 +00:00
Ben Johnson
e332f6fd7e
Merge pull request #1018 from molecula/rbf-cli 2020-10-28 06:26:27 -06:00