Commit graph

3 commits

Author SHA1 Message Date
Jason E. Aten
035073555a pilosa: only open views with data
- Previously, on timequantum schemas, we would
create and open a view for the cartesian
product of every possible view and shard.

- This caused us to be very slow on re-open,
and to use lots of memory for views that
held nothing.

- This change makes startup faster, memory
use much lower, and should speed migration.
2020-12-19 00:03:31 +00:00
Ben Johnson
150c8a5b06 database per shard, HolderConfig, rbf bit-wise import speedups.
- introduce Query Context (Qcx) for managing database-per-shard.
- replaces the MultiTx, so mtx.go is retired and removed.
- introduces the HolderConfig struct and all Holders now have
  a path from birth.
- rbf speedups on bitwise writes
- badgerdb is removed due to unresolvable write conflicts.

fixes #703 #676
2020-09-04 13:00:33 -05:00
Jason Aten
123ce41840 add lmdb, Tx call stats, and prep for db/shard.
- lmdb as a backend (lmdb.go)
   (lmdb is the fastest known transactional storage backend)
 - per Tx call statics report enabled with PILOSA_CALLSTAT=true (stattx.go)
 - framework for per-shard db (dbshard.go)
 - txfactory handles any pair under blue-green testing (txfactory.go)
 - enable CGO in Dockerfiles for lmdb
2020-08-17 18:26:58 -05:00