Commit graph

5920 commits

Author SHA1 Message Date
Cody Soyland
039780a85e
Merge pull request #685 from molecula/discard-test-386
no more test-386
2020-08-17 20:15:14 -05:00
Jason Aten
aea7d4a812 no more test-386
- lmdb won't build under 386
 - still have to keep lmdb_other.go for arm/arm64
2020-08-17 19:54:50 -05:00
jaten-molecula
ba1d5e9f9f
Merge pull request #681 from molecula/fix-rbf-reopen
Fix RBF checkpoint on reopen. Fixes #673
2020-08-17 19:16:17 -05:00
jaten-molecula
3e4da916e8
Merge branch 'master' into fix-rbf-reopen 2020-08-17 19:09:55 -05:00
tgruben
d84ae88944
Merge pull request #677 from molecula/lmdb_as_backend
add lmdb, Tx call stats, and prep for db/shard.
2020-08-17 18:43:57 -05:00
Jason Aten
123ce41840 add lmdb, Tx call stats, and prep for db/shard.
- lmdb as a backend (lmdb.go)
   (lmdb is the fastest known transactional storage backend)
 - per Tx call statics report enabled with PILOSA_CALLSTAT=true (stattx.go)
 - framework for per-shard db (dbshard.go)
 - txfactory handles any pair under blue-green testing (txfactory.go)
 - enable CGO in Dockerfiles for lmdb
2020-08-17 18:26:58 -05:00
seebs
4f7e4feaf8
Merge pull request #680 from seebs/cachefix
when updating a container, drop the single-container cache
2020-08-17 15:03:10 -05:00
Ben Johnson
b1e806a518 Fix RBF checkpoint on reopen.
Previously, the `rbf.DB.opened` flag was set after `checkpoint()`
when reopening, however, this flag is checked by `checkpoint()` so
it was not properly executing.
2020-08-17 13:28:28 -06:00
Seebs
16ba54293a when updating a container, drop the single-container cache
This can cause incredibly weird and hard-to-debug problems if the previous
container value is still in the cache after an update, and in particular,
can result in having a stale container value cached after a roaring import
that modified the container. Coupled with another bug which could corrupt
containers on a delete, this produces a very strange bug where a value is
present in a fragment, but an attempt to delete it reports failure.
2020-08-17 11:23:47 -05:00
Travis Turner
c5e9b8d917
Merge pull request #674 from travisturner/container-corrupt
stop setting N on a frozen container
2020-08-16 15:59:58 -05:00
Travis
ff5925aa8b
stop setting N on a frozen container 2020-08-14 17:25:14 -05:00
Matthew Jaffee
cd83f7965c
Merge pull request #672 from molecula/remove-transaction.md
remove old transactions doc
2020-08-14 14:40:10 -05:00
Matt Jaffee
b11bdeab87
remove old transactions doc 2020-08-14 11:57:28 -05:00
jaten-molecula
05ca83c2f5
Merge pull request #669 from molecula/autocommit2
build-tag out lmdb-go from builds; lower autocommit limits to avoid ErrTxnTooBig; improve txn size estimation
2020-08-13 11:52:42 -05:00
Jason Aten
c90c4c275d lower autocommit limits to avoid ErrTxnTooBig; improve txn size estimation 2020-08-13 16:42:47 +00:00
Travis Turner
1f53e86e47
Merge pull request #662 from travisturner/transaction-list
transactions as a list endpoint; added Transaction.CreatedAt
2020-08-13 08:40:57 -05:00
Travis
1837811ce1
transactions as a list endpoint; added Transaction.CreatedAt 2020-08-12 23:11:54 -05:00
tgruben
0dea018fb7
Merge pull request #665 from molecula/autocommit
tested working autocommit approach
2020-08-12 21:56:03 -05:00
Jason Aten
5e49e10cda tested working autocommit approach 2020-08-13 01:12:14 +00:00
jaten-molecula
c7d6229380
Merge pull request #664 from molecula/bigtx
PutContainer() handles large txn with autocommit
2020-08-12 19:18:19 -05:00
Jason Aten
6dc9727796 PutContainer() handles large txn with autocommit 2020-08-12 23:15:23 +00:00
jaten-molecula
973e9a7955
Merge pull request #659 from molecula/rbf_99pct
all test green on rbf. WOOT.
2020-08-12 16:13:47 -05:00
Todd Gruben
547ee14f5b all test green on rbf. WOOT.
- rbf had races around the new rootRecords cache in tx
 - rbf tx needed a write lock on the db now that rootRecords are written
 - added a global registry for rbfDB to correctly dedup instances
 - implement DeleteFragment, DeleteIndex for rbf
 - use badger style keys for rbf to allow content checksumming to be list
    containers in the same order
 - lots of other integration of rbf into pilosa layer.
2020-08-12 21:08:30 +00:00
Cody Soyland
123e0b6376
Merge pull request #644 from codysoyland/grpc-crd-sql
Add gRPC methods to support the functionality of VDSM
2020-08-12 14:55:24 -05:00
Cody Soyland
21456827ae
Merge branch 'master' into grpc-crd-sql 2020-08-12 14:21:53 -05:00
Cody Soyland
3361033c5b Add PostVDS and DeleteVDS 2020-08-12 14:21:22 -05:00
Cody Soyland
3771bd43b5 Add note about gRPC unary methods and futures 2020-08-12 14:08:30 -05:00
Nia
50a2e3c5fa
Merge pull request #650 from jaddr2line/limit
Add a "Limit" query
2020-08-12 14:04:12 -04:00
Nia
911c039991
Fix incorrect negative in Limit query documentation
Co-authored-by: Travis Turner <travis@pilosa.com>
2020-08-12 13:53:57 -04:00
Jaden Weiss
40a6fbd79e
add a "Limit" query 2020-08-12 12:27:23 -04:00
Nia
2a1fafb227
Merge pull request #652 from jaddr2line/constrow
Add ConstRow query
2020-08-12 12:22:06 -04:00
Nia
a19ca16810
Merge branch 'master' into constrow 2020-08-12 11:47:46 -04:00
Travis Turner
683b8dfe31
Merge pull request #660 from travisturner/error-rows-int
error on Rows(field=<int>)
2020-08-11 19:44:46 -05:00
Travis
eae038a07e
error on Rows(field=<int>) 2020-08-11 18:07:03 -05:00
Jaden Weiss
0e16192aeb
add ConstRow query 2020-08-11 08:25:32 -04:00
Jaden Weiss
96c2364c1b
Merge pull request #642 from jaddr2line/query-extract
Add an "Extract" query
2020-08-10 19:32:36 -04:00
Jaden Weiss
590bd07995
add an "Extract" query 2020-08-10 09:33:55 -04:00
Cody Soyland
559a2c6864 Add GetVDS and GetVDSs gRPC implementations 2020-08-07 16:28:06 -05:00
Ben Johnson
2bc3442ae8
Merge pull request #649 from molecula/fix-rbf-rollback
Fix RBF checkpoint high water mark
2020-08-07 14:29:31 -06:00
Ben Johnson
faa1662bf6 Fix RBF checkpoint high water mark
Previously, the `checkpoint()` function determined the segments to drop
based on the current active transactions' WAL ID references. However, if
no transactions are active then the checkpoint would drop segments too
aggressively.

This changes the determination by using the highest WAL ID that is
actually checkpointed to disk to determine the high water mark. If no
page are checkpointed then no segments can be dropped.
2020-08-07 10:07:33 -06:00
Ben Johnson
5644867507
Merge pull request #643 from molecula/fix-rbf-hanging-tests
Fix RBF/pilosa hanging tests
2020-08-07 10:06:57 -06:00
Cody Soyland
35675a3e59 Add stubbed gRPC types to support python-molecula
This adds the following rpc calls: GetVDS, GetVDSs, PostVDS, DeleteVDS, QuerySQL, QuerySQLUnary

Currently, they are not implemented.
2020-08-06 14:19:16 -05:00
Ben Johnson
78f9fbe332 Fix RBF/pilosa hanging tests 2020-08-06 11:43:41 -06:00
alanbernstein
5306426b19
Merge pull request #641 from alanbernstein/more-docs-updates
Fix various docs issues
2020-08-05 12:34:57 -05:00
Alan Bernstein
5f26290f92 Fix various docs issues 2020-08-05 12:26:22 -05:00
Ben Johnson
3fb2cccb00
Merge pull request #637 from molecula/rbf-fixes
Multiple RBF test fixes
2020-08-05 09:02:06 -06:00
Ben Johnson
f8cacd8081 Multiple RBF test fixes 2020-08-05 08:20:34 -06:00
Travis Turner
df709af10e
Merge pull request #638 from travisturner/fix-executeclearrow
safe cast of bool in executeClearRow
2020-08-04 15:52:48 -05:00
Travis
ddee2cb0a8
safe cast of bool in executeClearRow 2020-08-04 15:16:23 -05:00
tgruben
7f6c6956d8
Merge pull request #635 from molecula/efence_off
DetectMemAccessPastTx flag added, default false.
2020-08-03 22:03:36 -05:00