scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-10-09 12:27:52 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
No description
9426 commits 45 branches 174 tags 85 MiB
Find a file
Seebs e3d137f29c initial implementation of QueryContext design 
This is living in a subdirectory for now so we can have better
turnaround time on tests and not have to build everything else
along with it.

This covers the logic that we can have *without* actually using
databases or the filesystem in any way, just to provide a framework
that lets us validate the logic handling overlapping queries.

The overall purpose of this is to prevent deadlocks, by ensuring
that database locks are only taken when we have already proven
that they are available. In short, the QueryContext preregisters
its "scope" -- the set of things it may want to lock. The operation
of creating the QueryContext can block, but it blocks with no
database locks held. Once it is unblocked, the scope it has reported
is now considered unavailable, and no other QueryContext using any
overlapping scope can complete creation until this QueryContext
completes. While it's running, the QueryContext can't request write
access to anything outside its scope. Thus, once created, a
QueryContext can always proceed, without being blocked, until it's
done.

Note that this does not fully address multi-node behaviors;
once you have a QueryContext blocking things, you need to not
make queries to other nodes that could be blocked in turn by those
nodes. In short, no write queries to other nodes while holding a
write-type QueryContext on the local node, because if two nodes
do that to each other at once, they can both be blocked.

We believe RBF is currently designed such that read-only accesses
don't block progress on writes, so non-write access doesn't
create problems.

We also have some code to allow us to create dot-format output
from the components of this system, which is mostly intended to
be a debugging tool.
2022-10-28 14:01:58 -05:00
.github remove references to CONTRIBUTING and CHANGELOG 2021-11-19 11:22:15 -06:00
.gitlab Stop sending roaring-migrate binaries to S3 (#2263) 2022-10-27 17:25:29 -06:00
api/client Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
authn [FB-1621] account for token expiration as float and int (#2188) 2022-08-10 17:26:10 -05:00
authz drop ioutil 2022-09-23 16:56:27 -05:00
boltdb change perms: 0666 -> 0600; 0777 -> 0750 2022-07-21 13:34:46 -05:00
client resolving bool null field ingestion error (#2254) 2022-10-20 16:51:21 -05:00
cmd drop ioutil 2022-09-23 16:56:27 -05:00
ctl drop ioutil 2022-09-23 16:56:27 -05:00
debugstats add copyright notice back in 2021-12-10 11:01:04 -06:00
disco fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
encoding/proto fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
errors added /sql endpoint; implemented SHOW TABLES (#1935) 2022-08-31 13:01:05 -07:00
etcd fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
gcnotify Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
generator add copyright notice back in 2021-12-10 11:01:04 -06:00
gopsutil Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
hash drop ioutil 2022-09-23 16:56:27 -05:00
idk resolving bool null field ingestion error (#2254) 2022-10-20 16:51:21 -05:00
ingest drop ioutil 2022-09-23 16:56:27 -05:00
ingest_testdata allow ingest API delete to work at all, add test for it 2022-04-01 09:48:51 -05:00
install Change featurebase user home path to /var/lib/featurebase. (#2242) 2022-10-06 08:38:14 -07:00
internal Distinguish between three copies of fakeidp service (#2192) 2022-08-16 09:13:11 -05:00
lattice update molecula references to featurebase (#2262) 2022-10-27 15:57:53 -05:00
logger drop ioutil 2022-09-23 16:56:27 -05:00
lru fix some staticcheck issues 2021-12-03 09:31:45 -06:00
mock Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
monitor Add proper version info to sentry logs 2022-06-21 11:03:22 -05:00
net add copyright notice back in 2021-12-10 11:01:04 -06:00
pb fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
pql drop ioutil 2022-09-23 16:56:27 -05:00
prometheus Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
proto fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
qa make perf_able run at all, make it debug a bit better 2022-10-14 09:49:19 -05:00
querycontext initial implementation of QueryContext design 2022-10-28 14:01:58 -05:00
rbf don't obtain stack traces on rbf.Tx creation 2022-10-25 14:19:25 -05:00
roaring remove a commented-out test case for a function that no longer exists 2022-09-23 16:56:27 -05:00
scripts remove trailing spaces en masse (#2024) 2022-04-18 09:31:35 -05:00
server fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
shardwidth Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
short_txkey remove a bunch of commented print statements and unecessary prints 2022-01-11 10:42:44 -06:00
sql use testhook test cleanup 2022-09-26 12:33:43 -05:00
sql3 added a test to cover the keyword replace as being synonymous with insert (#2261) 2022-10-27 14:29:10 -05:00
statik Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
stats use testhook test cleanup 2022-09-26 12:33:43 -05:00
statsd Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
storage remove a bunch of roaring backend stuff 2022-02-02 20:56:18 -06:00
syswrap add copyright notice back in 2021-12-10 11:01:04 -06:00
task implement a task pool 2022-02-14 09:56:19 -06:00
test fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
testdata refactor testing to share clusters more often 2022-09-02 11:40:37 -05:00
testhook drop ioutil 2022-09-23 16:56:27 -05:00
toml add copyright notice back in 2021-12-10 11:01:04 -06:00
tracing Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
txkey remove a bunch of commented print statements and unecessary prints 2022-01-11 10:42:44 -06:00
vprint add copyright notice back in 2021-12-10 11:01:04 -06:00
.dockerignore FB-1597: unifying idk and featurebase (#2160) 2022-07-28 17:23:16 -05:00
.gitignore Fix some issues with running IDK tests in docker. (#2248) 2022-10-14 14:02:36 -05:00
.golangci.yml fix govet and gofmt errors in existing code 2021-10-29 13:14:27 -05:00
api.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
api_test.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
apimethod_string.go Remove most of the resize-related logic 2022-06-21 17:03:09 -05:00
audit.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
audit_internal_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
audit_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
broadcast.go simplify, streamline, and possibly debug embedded etcd 2022-07-21 11:42:35 -05:00
bsi.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
bsi_test.go add copyright notice back in 2021-12-10 11:01:04 -06:00
cache.go comment cleanup 2022-05-27 11:25:17 -05:00
cache_test.go Periodically invalidate rank cache during bulk add 2022-02-15 08:25:45 -07:00
catcher.go create BitmapRewriter/ApplyRewriter, parallel to BitmapFilter 2022-04-12 12:24:22 -05:00
cluster.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
cluster_internal_test.go refactor testing to share clusters more often 2022-09-02 11:40:37 -05:00
cmd.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
CODE_OF_CONDUCT.md chore(community): adds code of conduct (#2195) 2022-08-17 12:30:22 -04:00
const_amd64.go add copyright notice back in 2021-12-10 11:01:04 -06:00
const_other.go add copyright notice back in 2021-12-10 11:01:04 -06:00
dbshard.go drop the bool from a map that's only using it to check presence 2022-06-09 10:26:19 -05:00
dbshard_internal_test.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
dbshard_test.go refactor testing to share clusters more often 2022-09-02 11:40:37 -05:00
delete_test.go refactor testing to share clusters more often 2022-09-02 11:40:37 -05:00
diagnostics.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
diagnostics_internal_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
doc.go add copyright notice back in 2021-12-10 11:01:04 -06:00
Dockerfile Fix some issues with running IDK tests in docker. (#2248) 2022-10-14 14:02:36 -05:00
Dockerfile-clustertests Upgrade Go to 1.19 (#2204) 2022-08-25 14:54:14 -06:00
Dockerfile-clustertests-client Upgrade Go to 1.19 (#2204) 2022-08-25 14:54:14 -06:00
event.go simplify, streamline, and possibly debug embedded etcd 2022-07-21 11:42:35 -05:00
executor.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
executor_internal_test.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
executor_test.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
field.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
field_internal_test.go address multiple staticcheck issues 2022-09-23 16:56:27 -05:00
field_test.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
filesystem.go add copyright notice back in 2021-12-10 11:01:04 -06:00
fragment.go drop unused row 2022-09-23 16:56:27 -05:00
fragment_internal_test.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
gc.go add copyright notice back in 2021-12-10 11:01:04 -06:00
gid.go database per shard, HolderConfig, rbf bit-wise import speedups. 2020-09-04 13:00:33 -05:00
go.mod fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
go.sum Fix UUID library version for CVE-2021-3538 (#2213) 2022-09-06 13:52:13 -06:00
hack.go Change Ttl to TTL (#2038) 2022-04-27 11:20:39 -05:00
handler.go add doc comments 2022-05-27 11:25:17 -05:00
holder.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
holder_internal_test.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
holder_test.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
http_handler.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
http_handler_internal_test.go drop ioutil 2022-09-23 16:56:27 -05:00
http_handler_test.go address multiple staticcheck issues 2022-09-23 16:56:27 -05:00
http_translator.go drop ioutil 2022-09-23 16:56:27 -05:00
http_translator_test.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
idalloc.go change perms: 0666 -> 0600; 0777 -> 0750 2022-07-21 13:34:46 -05:00
idalloc_test.go change perms: 0666 -> 0600; 0777 -> 0750 2022-07-21 13:34:46 -05:00
index.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
index_internal_test.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
index_test.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
ingest_test.go drop ioutil 2022-09-23 16:56:27 -05:00
internal_client.go drop ioutil 2022-09-23 16:56:27 -05:00
internal_client_test.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
iterator.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
iterator_internal_test.go add copyright notice back in 2021-12-10 11:01:04 -06:00
license.exceptions update license header check exceptions 2021-05-14 10:45:46 -04:00
like.go add copyright notice back in 2021-12-10 11:01:04 -06:00
like_test.go add copyright notice back in 2021-12-10 11:01:04 -06:00
main_test.go use testhook test cleanup 2022-09-26 12:33:43 -05:00
Makefile fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
metrics.go add copyright notice back in 2021-12-10 11:01:04 -06:00
nfpm.yaml Fixed broken fields when packaging rpm and deb files. (#2153) 2022-07-27 12:54:46 -05:00
NOTICE add config files for release 2021-11-19 01:26:18 +03:00
pilosa.go Remove most of the resize-related logic 2022-06-21 17:03:09 -05:00
pilosa_internal_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
pilosa_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
pprof.go remove a bunch of roaring backend stuff 2022-02-02 20:56:18 -06:00
rbf.go address multiple staticcheck issues 2022-09-23 16:56:27 -05:00
README.md chore(community): adds code of conduct (#2195) 2022-08-17 12:30:22 -04:00
row.go remove string keys on delete to allow for reuse 2022-02-25 16:23:54 -06:00
row_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
serializer.go add copyright notice back in 2021-12-10 11:01:04 -06:00
server.go fb-1729 Enriched Table Metadata (#2255) 2022-10-26 11:23:40 -05:00
server_internal_test.go significant refactor of test setup and teardown 2022-09-23 16:56:27 -05:00
server_test.go refactor testing to share clusters more often 2022-09-02 11:40:37 -05:00
stattx.go create BitmapRewriter/ApplyRewriter, parallel to BitmapFilter 2022-04-12 12:24:22 -05:00
time.go FB-1456 - TTL - fixed views not returning correct results (#2062) 2022-05-16 15:26:13 -05:00
time_internal_test.go FB-1456 - TTL - fixed views not returning correct results (#2062) 2022-05-16 15:26:13 -05:00
tracker.go add copyright notice back in 2021-12-10 11:01:04 -06:00
tracker_test.go add copyright notice back in 2021-12-10 11:01:04 -06:00
transaction.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
transaction_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
translate.go drop ioutil 2022-09-23 16:56:27 -05:00
translator_test.go refactor testing to share clusters more often 2022-09-02 11:40:37 -05:00
tx.go create BitmapRewriter/ApplyRewriter, parallel to BitmapFilter 2022-04-12 12:24:22 -05:00
tx_internal_test.go significant refactor of test setup and teardown 2022-09-23 16:56:27 -05:00
tx_test.go use t.Fatal(f) to abort tests, not panic 2022-10-14 09:49:19 -05:00
txfactory.go address multiple staticcheck issues 2022-09-23 16:56:27 -05:00
txfactory_internal_test.go remove a bunch of roaring backend stuff 2022-02-02 20:56:18 -06:00
util.go continue removing Tx parameters to view-type functions 2022-09-23 16:56:27 -05:00
util_test.go [FB-1024 FB-1590] Increase timerange (un-revert) (#2174) 2022-08-04 15:20:45 -07:00
utils_internal_test.go refactor testing to share clusters more often 2022-09-02 11:40:37 -05:00
version.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
view.go significant refactor of test setup and teardown 2022-09-23 16:56:27 -05:00
view_internal_test.go significant refactor of test setup and teardown 2022-09-23 16:56:27 -05:00

README.md

FeatureBase, a distributed bitmap index

Contributor Covenant Coverage SecurityRating Quality Gate Status

See our internal documentation, which includes all external documentation, plus many internal-only pages, listed under the "Internal" heading in the main navigation bar.

Follow along with the Sample Project to get a better understanding of FeatureBase's capabilities.