Commit graph

20 commits

Author SHA1 Message Date
Seebs
6af987a5ba fix error formatting/spelling
Go convention is that error messages don't end with periods and
don't start with capital letters.
2021-08-18 13:45:36 -05:00
Mahesh Arumugam
858f889745 FeatureBase Renaming: changing go.mod module name for featurebase 2021-07-19 09:20:30 -07:00
Todd Gruben
9d24fb07b7 wired in restore command 2021-05-21 09:27:08 -05:00
Seebs
5ffa4ba803 drop "batched" flag from Add operation
The "batched" flag creates a complexity which is that the return value of Add
might or might not be meaningful, but it doesn't really buy us very much.

If we are concerned about the ops log size of writing single ops as 21-byte
arrays of 1 op rather than as 13-byte ops, we can make the AddN code smarter
about how it writes ops. And probably should.

Along with this, change Remove to use the batched operation form, which
writes a more meaningful ops log, and return a meaningful value for changes
made. Otherwise, it ends up writing potentially thousands of ops to the
ops log without reporting any OpN, because the number of ops written isn't
the same as the number of changes those ops made. This could result in
files growing by megabytes without OpN changing.

There was a comment here about a test failing with RemoveN. I can't prove
it, but I strongly suspect that this was actually a result of that test
case hitting a particular bug that we eventually fixed, and which we might
have fixed sooner if we'd realized why using RemoveN made that test
fail.
2021-04-20 12:03:25 -05:00
Kuba Podgórski
9a02004a4f Move vprint to separate pakage 2021-03-29 14:29:19 +02:00
Travis
f78a6508f0
Remove stutter and suffix from backend files 2021-03-07 22:11:51 -06:00
Travis
4661f3ac08
reorganize the storage backends directory 2021-03-05 16:56:44 -06:00
Travis
d192c1f24f
Merge branch 'master' into disco 2021-02-05 15:58:36 -06:00
Alan Bernstein
e397d35ed5 Include roaring field and key details in usage endpoint 2021-01-29 17:48:24 -06:00
Travis
08fae2be4c
introduce storage.Config 2021-01-20 22:05:38 -06:00
Jason E. Aten
035073555a pilosa: only open views with data
- Previously, on timequantum schemas, we would
create and open a view for the cartesian
product of every possible view and shard.

- This caused us to be very slow on re-open,
and to use lots of memory for views that
held nothing.

- This change makes startup faster, memory
use much lower, and should speed migration.
2020-12-19 00:03:31 +00:00
Seebs
30d5f891c0 create naive ApplyFilter 2020-12-16 13:16:46 -06:00
Jason E. Aten
b52a814b3c short_txkey elides index and shard from the txkey
- use short_txkey for rbf
 - short_txkey breaks a bunch of bolt_test.go, so leave it on (long) txkey for now.
 - remove SliceOfShards method from Tx interface
2020-12-11 22:24:56 +00:00
Jason E. Aten
7a9e0969cb remove lmdb as a Tx backend
- test only still uses a Barrier utility from the go-lmdb package;
  it could be ported in at some point.
2020-12-04 23:17:59 +00:00
Jason E. Aten
a38d4fa46c apply --fsync flag to all tx backend
- rename from --rbf-fsync to --fsync, as it
   now effects bolt, lmdb too.
2020-11-06 18:11:02 +00:00
Jason E. Aten
9c66cd5a81 DeleteEmptyContainer true by default now 2020-10-30 04:15:53 +00:00
Jason E. Aten
509348260f better migration logging 2020-10-28 22:54:58 +00:00
Jason E. Aten
957cba1768 performance tuning: rbfcfg package, binary search for wal segment
- rbfcfg package holds Config for --rbf- command line flags
- wal.go: replace linear search with bisection for wal segment
2020-10-27 00:49:56 +00:00
Jason E. Aten
8d2ad048fa pilosa server --norowcache disables the row cache
- this can lessen memory pressure
- certain backends may not need it
- enables performance benchmarking and tuning
2020-10-22 22:20:48 +00:00
Jason E. Aten
266b92c025 Use boltdb instead of badger as our all Go Tx oracle.
- remove all badgerdb code.
 - use boltdb instead.
2020-10-16 17:21:21 -05:00