scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-10-11 05:17:54 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
9444 commits 45 branches 174 tags 85 MiB
Commit graph

9444 commits

This branch
This branch
All branches
Author SHA1 Message Date
Seebs
7c415b4217 Implement rbf-specific ApplyFilter 
This gives RBF an ApplyFilter that can run without instantiating containers
when the filter it's using doesn't need them instantiated. We can also seek
ahead in cases where we know the next key we care about is not just the next
key numerically.
 2020-12-16 13:16:46 -06:00
Seebs
2c4aff2d18 add benchmarks for f.rows() 
This is a simplistic benchmark for f.rows() to let us evaluate its performance
in preparation for trying to do some profiling and tuning.
 2020-12-16 13:16:46 -06:00
Seebs
30d5f891c0 create naive ApplyFilter 2020-12-16 13:16:46 -06:00
Seebs
59d89dda99 Allow arbitrary and potentially more efficient filtering of bitmaps 
This is a partial solution to a nasty performance problem, which is that
a ContainerIterator has to *generate* all the containers. With roaring, this
was cheap because they already exist in memory; with transactional backends,
it's an allocation per container, *even for the containers we don't use*.

This design admits filters which can distinguish between answers they
can give just based on keys and times when they actually need containers
instantiated, and can also give hints as to future answers -- saying "yes"
or "no" to entire rows at a time, or indicating when they're done.

This is only part of the solution; we also need a Tx API hook for
doing scans like this which doesn't rely on ContainerIterator.
 2020-12-16 13:16:46 -06:00
jaten-molecula
440b90f3c8
Merge pull request #1240 from jaten-molecula/roaring_printutil 
roaring: print utility AsContainerMatrixString for diagnostics
 2020-12-16 13:02:47 -06:00
Jason E. Aten
1a6f573739 roaring: print utility AsContainerMatrixString for diagnostics 
AsContainerMatrixString returns a string showing
 the matrix of rows in a shard, showing the count of hot (1) bits
 in each container.
 2020-12-16 18:50:45 +00:00
Nia Weiss
eab70e2314
use the txf to determine if snapshots are needed 2020-12-16 11:00:27 -05:00
Nia Weiss
aaf94c6aeb
fix the automatic snapshot queue enable check 
The "needs snapshot queue" check was broken, as it only checked inside a loop over indices.
If there are no indexes yet (or more likely if the indices have not yet been loaded off of disk), then this would never use the snapshot queue on roaring.
 2020-12-16 08:15:17 -05:00
Cody Soyland
87bf0df786
Merge pull request #1234 from codysoyland/rbf-usage-calc 
Fix disk usage calculation in RBF backend
 2020-12-15 15:57:17 -06:00
Cody Soyland
4cb21da6e9 Fix disk usage calculation in RBF backend 2020-12-15 13:39:27 -06:00
Cody Soyland
765bac992d
Merge pull request #1225 from codysoyland/upgrade-gopsutil 
Upgrade gopsutil
 2020-12-14 11:36:07 -06:00
Cody Soyland
5eb726ce59 Upgrade gopsutil 2020-12-14 10:58:28 -06:00
tgruben
3beaed1645
Merge pull request #1220 from jaten-molecula/avoid_dynamic_dispatch 
FragSpec as struct, fragment fields to avoid dynamic dispatch
 2020-12-14 09:21:57 -06:00
Jason E. Aten
b254c6776b FragSpec as struct, fragment fields to avoid dynamic dispatch 
- allow inlining of getters index(), view().
 - GOMAXPROCS set to 128
 2020-12-12 16:39:24 +00:00
tgruben
b0755577b3
Merge pull request #1218 from jaten-molecula/short_txkey 
short_txkey elides index and shard from the txkey
 2020-12-11 17:30:40 -06:00
Jason E. Aten
b52a814b3c short_txkey elides index and shard from the txkey 
- use short_txkey for rbf
 - short_txkey breaks a bunch of bolt_test.go, so leave it on (long) txkey for now.
 - remove SliceOfShards method from Tx interface
 2020-12-11 22:24:56 +00:00
tgruben
c5302f2f57
Merge pull request #1212 from tgruben/fragment-refactor 
Fragment refactor (wip)
 2020-12-11 15:08:19 -06:00
Todd Gruben
8ad7afbe43 FragProxy reduces string memory consumption drastically 
for datasets with lots of views, because we don't
replicate path, index, field, view strings so often.
 2020-12-11 21:01:15 +00:00
jaten-molecula
30502c99ab
Merge pull request #1214 from jaten-molecula/readwrite_locks 
use tx.mu.RLock for OffsetRange, Lock for AddRoaring
 2020-12-11 08:13:52 -06:00
Jason E. Aten
2b569b1edf use tx.mu.RLock for OffsetRange, Lock for AddRoaring 2020-12-11 13:53:27 +00:00
tgruben
f5e656138f
Merge pull request #1211 from jaten-molecula/skip_serz_1node 
avoid serializing for sync shards in 1 node cluster situation
 2020-12-10 17:16:10 -06:00
Jason E. Aten
7571792401 avoid serializing for sync shards in 1 node cluster situation 2020-12-10 20:54:32 +00:00
Maxton Huff
8c20255b12
Merge pull request #1210 from Maxtonian/lonquerytime2 
deprecate cluster.long-query-time and create long-query-time
 2020-12-10 11:14:09 -06:00
Maxton Huff
dee700741a deprecate cluster.long-query-time and create long-query-time 
moved lonquerytime from cluster into server and moved cluster.longquerytime into top level config
kept cluster.longquerytime for backwards compatibility, favored if both longquerytime options are present
 2020-12-10 10:51:31 -06:00
tgruben
087418566b
Merge pull request #1204 from jaten-molecula/clear_cursors 
reset cursor stacks before reuse
 2020-12-09 19:01:08 -06:00
jaten-molecula
21cb390d2f
Merge branch 'master' into clear_cursors 2020-12-09 17:01:21 -06:00
Jason E. Aten
81b9586a4b reset cursor stacks before reuse 2020-12-09 21:41:41 +00:00
Cody Soyland
1990ecf2b6
Merge pull request #1205 from codysoyland/docker-build-make 
Use "make" inside docker build so that CGO_ENABLED uses exported value from Makefile
 2020-12-09 15:02:04 -06:00
Cody Soyland
f351b7efa6 Use "make" inside docker build so that CGO_ENABLED uses exported value from Makefile 2020-12-09 13:21:07 -06:00
Cody Soyland
1add244f8a
Merge pull request #1201 from codysoyland/default-roaring 
Default TxSrc to roaring
 2020-12-09 09:36:28 -06:00
Cody Soyland
c1fe8b214a Default TxSrc to roaring 2020-12-09 09:31:40 -06:00
Cody Soyland
53f6a127af
Merge pull request #1198 from codysoyland/docker-build-test 
Disable cgo on all builds, add additional tests for docker
 2020-12-09 09:17:03 -06:00
Cody Soyland
5d18c0af2b Enable cgo on test -race 2020-12-09 09:00:03 -06:00
Cody Soyland
bc94c7bcdf Remove lmdb dependency and references, vendor Barrier 2020-12-09 08:43:23 -06:00
Cody Soyland
eef6359db3 Disable cgo on all builds, add additional tests for docker 2020-12-08 19:42:51 -06:00
tgruben
d05531b77a
Merge pull request #1197 from jaten-molecula/rowcache_off_by_default 
rowcache off by default. pilosa server --rowcache-on turns it back on.
 2020-12-08 19:32:19 -06:00
Jason E. Aten
03a54c6d5f rowcache off by default. pilosa server --rowcache-on turns it back on. 2020-12-08 23:18:45 +00:00
tgruben
d4d30e160b
Merge pull request #1196 from jaten-molecula/diagn 
pilosa debugstats and rbf tooling for enhanced debugging/diagnostics
 2020-12-08 16:58:54 -06:00
Jason E. Aten
82d07bc123 debugstats and rbf tooling for enhanced debugging/diagnostics 2020-12-08 22:47:40 +00:00
tgruben
9f7fa7eaad
Merge pull request #1195 from jaten-molecula/rbf_doc 
document pattern of branch splits
 2020-12-08 16:19:56 -06:00
tgruben
63d67ac65e
Merge branch 'master' into rbf_doc 2020-12-08 15:52:46 -06:00
tgruben
662073c284
Merge pull request #1194 from jaten-molecula/fix_freepageset 
fix bug in pgno computation in freePageSet
 2020-12-08 15:52:25 -06:00
Jason E. Aten
cbff5bd29d document pattern of branch splits 2020-12-08 20:50:29 +00:00
Jason E. Aten
96abbfa059 fix bug in pgno computation in freePageSet 2020-12-08 20:45:54 +00:00
Cody Soyland
bd060b6fd2
Merge pull request #1191 from codysoyland/docker-disable-cgo 
Dockerfile: switch back to alpine and disable cgo
 2020-12-07 10:07:56 -06:00
Cody Soyland
2bd27cdf5e Upgrade alpine and disable cgo 2020-12-07 09:26:52 -06:00
Cody Soyland
5d08d54518 Revert "ubuntu 20:10 image instead of alpine, for cgo support" 
This reverts commit ef156f6172.
 2020-12-07 09:26:52 -06:00
Nia
8a6d20d314
Merge pull request #1168 from niaow/pilosa-id-gen 
Add ID auto-generation on the coordinator
 2020-12-07 09:58:13 -05:00
Nia
a4f538ffec
Merge branch 'master' into pilosa-id-gen 2020-12-07 09:46:58 -05:00
Nia
2a863e5c13
clarify OptServerOpenIDAllocator 
Co-authored-by: Matthew Jaffee <matthew.jaffee@gmail.com>
 2020-12-07 08:08:51 -05:00