scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-08-28 10:54:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
No description
9252 commits 45 branches 174 tags 85 MiB
Find a file
Matthew Jaffee 772496b440 "all bitmap" multi-field, single-shard ingest 
This adds a shard-based import endpoint which takes bitmap data for
all field types and imports data for the whole shard transactionally.

It uses the BitmapRewriter interface to try to intelligently allow for
setting and clearing bits simultaneously without multiple writes which
is especially helpful when ingesting into int-like fields, but also
allows clear-and-then-set behavior for set fields.
2022-05-27 11:25:17 -05:00
.github remove references to CONTRIBUTING and CHANGELOG 2021-11-19 11:22:15 -06:00
.gitlab Smoketest cleanup (#2077) 2022-05-23 16:37:42 -04:00
api/client Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
authn [FB-1379] Create a featurebase subcommand to obtain an auth token (#2079) 2022-05-26 11:35:51 -05:00
authz Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
boltdb use free id bucket to re-use ids 2022-02-28 15:56:58 -06:00
client "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
cmd "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
ctl "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
debugstats add copyright notice back in 2021-12-10 11:01:04 -06:00
disco implement ability to update TTL on time fields 2022-05-04 11:04:38 -05:00
encoding/proto "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
etcd [FB-1398] Mainline etcd (#2049) 2022-05-11 09:49:48 -05:00
gcnotify Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
generator add copyright notice back in 2021-12-10 11:01:04 -06:00
gopsutil Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
hash Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
ingest Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
ingest_testdata allow ingest API delete to work at all, add test for it 2022-04-01 09:48:51 -05:00
install remove trailing spaces en masse (#2024) 2022-04-18 09:31:35 -05:00
internal Add refresh token header/cookie (#2071) 2022-05-20 16:12:27 -05:00
lattice remove trailing spaces en masse (#2024) 2022-04-18 09:31:35 -05:00
logger Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
lru fix some staticcheck issues 2021-12-03 09:31:45 -06:00
mock Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
net add copyright notice back in 2021-12-10 11:01:04 -06:00
pb "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
pg remove trailing spaces en masse (#2024) 2022-04-18 09:31:35 -05:00
pql FB-1459 ugly first cut at supportings Rows(in=[...]) (#2066) 2022-05-25 11:57:59 -05:00
prometheus Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
proto Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
qa "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
rbf "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
roaring "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
scripts remove trailing spaces en masse (#2024) 2022-04-18 09:31:35 -05:00
server Recognize semi-colon separated cookies too (#2078) 2022-05-24 21:45:33 -05:00
shardwidth Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
short_txkey remove a bunch of commented print statements and unecessary prints 2022-01-11 10:42:44 -06:00
sql gracefully handle invalid indexes in joins 2022-04-13 11:44:58 -05:00
sql2 Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
statik Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
stats only set bits after the holder is completely setup 2022-02-25 15:02:24 -06:00
statsd Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
storage remove a bunch of roaring backend stuff 2022-02-02 20:56:18 -06:00
syswrap add copyright notice back in 2021-12-10 11:01:04 -06:00
task implement a task pool 2022-02-14 09:56:19 -06:00
test [FB-1398] Mainline etcd (#2049) 2022-05-11 09:49:48 -05:00
testdata FB-1378: Use IP whitelisting for ingest authentication and authorization (#2070) 2022-05-20 14:57:05 -05:00
testhook Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
toml add copyright notice back in 2021-12-10 11:01:04 -06:00
topology add PartitionToNodeAssignment as a new option 2022-04-27 17:21:47 -05:00
tracing Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
txkey remove a bunch of commented print statements and unecessary prints 2022-01-11 10:42:44 -06:00
vprint add copyright notice back in 2021-12-10 11:01:04 -06:00
.dockerignore Add new Docker build process and continuous delivery 2021-02-26 14:34:14 -06:00
.gitignore remove trailing spaces en masse (#2024) 2022-04-18 09:31:35 -05:00
.golangci.yml fix govet and gofmt errors in existing code 2021-10-29 13:14:27 -05:00
api.go "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
api_test.go "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
apimethod_string.go FB-1188 - Added TTL field option 2022-02-25 11:39:31 -06:00
audit.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
audit_internal_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
audit_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
broadcast.go implement ability to update TTL on time fields 2022-05-04 11:04:38 -05:00
bsi.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
bsi_test.go add copyright notice back in 2021-12-10 11:01:04 -06:00
cache.go Periodically invalidate rank cache during bulk add 2022-02-15 08:25:45 -07:00
cache_test.go Periodically invalidate rank cache during bulk add 2022-02-15 08:25:45 -07:00
catcher.go create BitmapRewriter/ApplyRewriter, parallel to BitmapFilter 2022-04-12 12:24:22 -05:00
cluster.go implement ability to update TTL on time fields 2022-05-04 11:04:38 -05:00
cluster_internal_test.go add PartitionToNodeAssignment as a new option 2022-04-27 17:21:47 -05:00
cmd.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
const_amd64.go add copyright notice back in 2021-12-10 11:01:04 -06:00
const_other.go add copyright notice back in 2021-12-10 11:01:04 -06:00
dbshard.go remove unused stuff and fix a bunch of random staticcheck issues 2022-02-07 15:10:10 -06:00
dbshard_internal_test.go comment cleanup and removed long test 2022-02-25 16:40:04 -06:00
dbshard_test.go remove http subpackage and bring implementations into core 2022-02-03 21:04:04 -06:00
delete_test.go Add regression test for FB-1281 2022-03-24 15:20:57 -06:00
diagnostics.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
diagnostics_internal_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
doc.go add copyright notice back in 2021-12-10 11:01:04 -06:00
Dockerfile remove references to LICENSE and checks for it in source files 2021-11-19 10:38:06 -06:00
Dockerfile-clustertests remove trailing spaces en masse (#2024) 2022-04-18 09:31:35 -05:00
Dockerfile-clustertests-client remove trailing spaces en masse (#2024) 2022-04-18 09:31:35 -05:00
event.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
executor.go If the field doesn't exist return ErrFieldNotFound (#2081) 2022-05-26 16:45:10 -05:00
executor_internal_test.go track closing status for index/field/view, shut down cache flush early 2022-03-18 14:41:44 -05:00
executor_test.go If the field doesn't exist return ErrFieldNotFound (#2081) 2022-05-26 16:45:10 -05:00
field.go FB-1403 - fixed TTL to not allowed negative values (#2076) 2022-05-23 18:35:57 -05:00
field_internal_test.go dont default to standard view for some time range queries 2022-05-04 11:04:38 -05:00
field_test.go Change Ttl to TTL (#2038) 2022-04-27 11:20:39 -05:00
filesystem.go add copyright notice back in 2021-12-10 11:01:04 -06:00
fragment.go "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
fragment_internal_test.go track closing status for index/field/view, shut down cache flush early 2022-03-18 14:41:44 -05:00
gc.go add copyright notice back in 2021-12-10 11:01:04 -06:00
gid.go database per shard, HolderConfig, rbf bit-wise import speedups. 2020-09-04 13:00:33 -05:00
go.mod [FB-1388] Add optional support for continuous profiling in datadog (#2037) 2022-05-12 12:33:46 -05:00
go.sum [FB-1388] Add optional support for continuous profiling in datadog (#2037) 2022-05-12 12:33:46 -05:00
hack.go Change Ttl to TTL (#2038) 2022-04-27 11:20:39 -05:00
handler.go "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
holder.go add PartitionToNodeAssignment as a new option 2022-04-27 17:21:47 -05:00
holder_internal_test.go address reviewer's comments 2022-03-10 09:36:18 -06:00
holder_test.go change assignment of partitions to nodes 2022-04-27 17:21:47 -05:00
http_handler.go "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
http_handler_internal_test.go [FB-1379] Create a featurebase subcommand to obtain an auth token (#2079) 2022-05-26 11:35:51 -05:00
http_handler_test.go FB-1403 - fixed TTL to not allowed negative values (#2076) 2022-05-23 18:35:57 -05:00
http_translator.go remove http subpackage and bring implementations into core 2022-02-03 21:04:04 -06:00
http_translator_test.go remove http subpackage and bring implementations into core 2022-02-03 21:04:04 -06:00
idalloc.go add copyright notice back in 2021-12-10 11:01:04 -06:00
idalloc_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
index.go FB-1403 - fixed TTL to not allowed negative values (#2076) 2022-05-23 18:35:57 -05:00
index_internal_test.go remove unused stuff and fix a bunch of random staticcheck issues 2022-02-07 15:10:10 -06:00
index_test.go track closing status for index/field/view, shut down cache flush early 2022-03-18 14:41:44 -05:00
ingest_test.go allow ingest API delete to work at all, add test for it 2022-04-01 09:48:51 -05:00
internal_client.go [FB-1379] Create a featurebase subcommand to obtain an auth token (#2079) 2022-05-26 11:35:51 -05:00
internal_client_test.go Add refresh token header/cookie (#2071) 2022-05-20 16:12:27 -05:00
iterator.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
iterator_internal_test.go add copyright notice back in 2021-12-10 11:01:04 -06:00
license.exceptions update license header check exceptions 2021-05-14 10:45:46 -04:00
like.go add copyright notice back in 2021-12-10 11:01:04 -06:00
like_test.go add copyright notice back in 2021-12-10 11:01:04 -06:00
main_test.go listen on localhost:0 instead of :0 2022-03-18 13:38:30 -05:00
Makefile "all bitmap" multi-field, single-shard ingest 2022-05-27 11:25:17 -05:00
metrics.go add copyright notice back in 2021-12-10 11:01:04 -06:00
nfpm.yaml FB-904: Add deb/rpm packaging of FeatureBase binary 2021-11-04 10:32:59 -06:00
NOTICE add config files for release 2021-11-19 01:26:18 +03:00
pilosa.go test multiple field creations at once on a cluster 2022-03-04 10:21:18 -06:00
pilosa_internal_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
pilosa_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
planner.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
planner_test.go track closing status for index/field/view, shut down cache flush early 2022-03-18 14:41:44 -05:00
pprof.go remove a bunch of roaring backend stuff 2022-02-02 20:56:18 -06:00
rbf.go create BitmapRewriter/ApplyRewriter, parallel to BitmapFilter 2022-04-12 12:24:22 -05:00
README.md add coverage, code quality badges 2022-03-30 10:07:23 -05:00
row.go remove string keys on delete to allow for reuse 2022-02-25 16:23:54 -06:00
row_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
serializer.go add copyright notice back in 2021-12-10 11:01:04 -06:00
server.go FB-1403 - TTL - Added check for upper bound date (#2053) 2022-05-11 13:57:29 -05:00
server_internal_test.go improve Server WaitGroup concurrent usage 2022-02-25 15:02:21 -06:00
server_test.go FB-1456 - TTL - fixed views not returning correct results (#2062) 2022-05-16 15:26:13 -05:00
stattx.go create BitmapRewriter/ApplyRewriter, parallel to BitmapFilter 2022-04-12 12:24:22 -05:00
time.go FB-1456 - TTL - fixed views not returning correct results (#2062) 2022-05-16 15:26:13 -05:00
time_internal_test.go FB-1456 - TTL - fixed views not returning correct results (#2062) 2022-05-16 15:26:13 -05:00
tracker.go add copyright notice back in 2021-12-10 11:01:04 -06:00
tracker_test.go add copyright notice back in 2021-12-10 11:01:04 -06:00
transaction.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
transaction_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
translate.go remove string keys on delete to allow for reuse 2022-02-25 16:23:54 -06:00
translator_test.go remove http subpackage and bring implementations into core 2022-02-03 21:04:04 -06:00
tx.go create BitmapRewriter/ApplyRewriter, parallel to BitmapFilter 2022-04-12 12:24:22 -05:00
tx_internal_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
tx_test.go remove string keys on delete to allow for reuse 2022-02-25 16:23:54 -06:00
txfactory.go . 2022-03-11 06:20:07 -06:00
txfactory_internal_test.go remove a bunch of roaring backend stuff 2022-02-02 20:56:18 -06:00
util.go [FB-1379] Create a featurebase subcommand to obtain an auth token (#2079) 2022-05-26 11:35:51 -05:00
util_test.go add /internal/disk-usage endpoint for testing 2022-03-14 15:23:15 -05:00
utils_internal_test.go [FB-1379] Create a featurebase subcommand to obtain an auth token (#2079) 2022-05-26 11:35:51 -05:00
version.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
view.go track closing status for index/field/view, shut down cache flush early 2022-03-18 14:41:44 -05:00
view_internal_test.go Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00

README.md

FeatureBase, a distributed bitmap index

Coverage SecurityRating Quality Gate Status

See our internal documentation, which includes all external documentation, plus many internal-only pages, listed under the "Internal" heading in the main navigation bar.

Follow along with the Sample Project to get a better understanding of FeatureBase's capabilities.