Commit graph

4 commits

Author SHA1 Message Date
Ben Johnson
150c8a5b06 database per shard, HolderConfig, rbf bit-wise import speedups.
- introduce Query Context (Qcx) for managing database-per-shard.
- replaces the MultiTx, so mtx.go is retired and removed.
- introduces the HolderConfig struct and all Holders now have
  a path from birth.
- rbf speedups on bitwise writes
- badgerdb is removed due to unresolvable write conflicts.

fixes #703 #676
2020-09-04 13:00:33 -05:00
Jason Aten
123ce41840 add lmdb, Tx call stats, and prep for db/shard.
- lmdb as a backend (lmdb.go)
   (lmdb is the fastest known transactional storage backend)
 - per Tx call statics report enabled with PILOSA_CALLSTAT=true (stattx.go)
 - framework for per-shard db (dbshard.go)
 - txfactory handles any pair under blue-green testing (txfactory.go)
 - enable CGO in Dockerfiles for lmdb
2020-08-17 18:26:58 -05:00
Jason Aten
c90c4c275d lower autocommit limits to avoid ErrTxnTooBig; improve txn size estimation 2020-08-13 16:42:47 +00:00
Todd Gruben
547ee14f5b all test green on rbf. WOOT.
- rbf had races around the new rootRecords cache in tx
 - rbf tx needed a write lock on the db now that rootRecords are written
 - added a global registry for rbfDB to correctly dedup instances
 - implement DeleteFragment, DeleteIndex for rbf
 - use badger style keys for rbf to allow content checksumming to be list
    containers in the same order
 - lots of other integration of rbf into pilosa layer.
2020-08-12 21:08:30 +00:00