No description
Find a file
2023-04-04 08:38:32 -05:00
.github/workflows Increase golangci-lint timeout to 8 minutes 2023-03-17 09:59:09 -06:00
.gitlab handle tag-only commits with no hash in version 2023-03-30 17:00:22 -05:00
api/client enable staticcheck in ci; remove dead code (#2281) 2022-11-15 11:33:12 -08:00
authn Dataframe (#2241) 2022-12-12 09:01:20 -08:00
authz fix: updating code to meet linting requirements (#2171) 2022-09-29 12:34:29 -04:00
batch standardize existence-tracking logic a bit better 2023-03-24 16:01:09 -05:00
buffer limit memory for backuptar/restoretar (#2270) 2023-02-23 18:35:52 -06:00
bufferpool Enable linter: stylecheck (#2317) 2023-03-14 08:45:18 -05:00
cli fbsql: add the --csv and --pset flags (#2342) 2023-03-22 08:57:28 -05:00
client Support for setting individual DatabaseOptions (#2231) 2023-02-06 08:50:28 -06:00
cmd fbsql: add the --csv and --pset flags (#2342) 2023-03-22 08:57:28 -05:00
context implement fb_exec_requests system table (#2327) 2022-12-12 09:01:20 -08:00
ctl Sqldb rip boltdb (#2341) 2023-03-22 08:54:13 -05:00
dax hacky workaround: use locking to quiet race detector problems 2023-03-28 11:56:23 -05:00
debugstats linter: prealloc (#2315) 2023-03-11 21:19:05 -06:00
disco controller commit to DB first then send directives (#2259) 2023-02-17 09:12:15 -06:00
encoding/proto support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
errors Cloud 1358 bolt postgres (#2286) 2023-03-20 09:02:22 -05:00
etcd Enable linter: stylecheck (#2317) 2023-03-14 08:45:18 -05:00
extendiblehash fix import paths 2023-01-11 18:59:24 +00:00
gcnotify Updated dependency paths to reflect new repo location 2022-09-06 09:39:22 -07:00
generator Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
gopsutil Updated dependency paths to reflect new repo location 2022-09-06 09:39:22 -07:00
hash fix: updating code to meet linting requirements (#2171) 2022-09-29 12:34:29 -04:00
idk off-by-v error in spelling of versions 2023-03-30 17:34:59 -05:00
install Packaging improvements to improve mcloud updates (#2288) 2022-11-15 11:33:12 -08:00
internal Sup 294 pre sort command (#2209) 2023-01-23 12:26:38 -06:00
lattice Bump minimist from 1.2.5 to 1.2.8 in /lattice (#2289) 2023-03-03 16:09:14 -06:00
logger tweak a bunch of logging and config (#2234) 2023-02-03 14:59:07 -06:00
lru Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
mock Updated dependency paths to reflect new repo location 2022-09-06 09:39:22 -07:00
monitor Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
net staticcheck fixes (#2278) 2022-11-15 11:33:10 -08:00
pb support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
pql support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
prometheus Sup 294 pre sort command (#2209) 2023-01-23 12:26:38 -06:00
proto Sup 294 pre sort command (#2209) 2023-01-23 12:26:38 -06:00
qa Rename some interfaces. Remove the ComputeAPI (#2333) 2022-12-12 09:01:20 -08:00
querycontext bump timeout on overlapping write requests test 2022-12-12 09:01:20 -08:00
rbf linter: prealloc (#2315) 2023-03-11 21:19:05 -06:00
roaring Enable linter: stylecheck (#2317) 2023-03-14 08:45:18 -05:00
runners feat: builds docker images for running featurebase from a release (#2193) 2022-11-08 13:07:17 -05:00
scripts Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
server support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
shardwidth feat: updates shardwidth.Exponent to single constant (#2185) 2022-10-25 13:30:00 -04:00
short_txkey staticcheck fixes (#2278) 2022-11-15 11:33:10 -08:00
sql Enable linter: stylecheck (#2317) 2023-03-14 08:45:18 -05:00
sql3 fb-2040 (#2354) 2023-03-28 16:59:00 -04:00
statik staticcheck fixes (#2278) 2022-11-15 11:33:10 -08:00
stats introduce performance counters and system table fanout, plus refactor metrics (#2363) 2023-01-19 21:35:02 +00:00
storage Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
systemlayer try to make sure we're releasing large allocations (#2225) 2023-01-25 14:48:01 -06:00
syswrap Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
task task pool: avoid race condition on shutdown/close 2023-02-17 11:32:07 -06:00
test Enable linter: stylecheck (#2317) 2023-03-14 08:45:18 -05:00
testdata refactor testing to share clusters more often 2022-09-30 11:10:47 -07:00
testhook Enable linter: stylecheck (#2317) 2023-03-14 08:45:18 -05:00
toml Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
tracing Updated dependency paths to reflect new repo location 2022-09-06 09:39:22 -07:00
txkey staticcheck fixes (#2278) 2022-11-15 11:33:10 -08:00
vprint Enable linter: stylecheck (#2317) 2023-03-14 08:45:18 -05:00
wireprotocol SQL3 tests newMessageError and all the constructors (#2320) 2023-03-15 12:41:35 -05:00
.gitignore Cloud 1358 bolt postgres (#2286) 2023-03-20 09:02:22 -05:00
.golangci.yml Enable linter: stylecheck (#2317) 2023-03-14 08:45:18 -05:00
api.go rework and consolidate view name cleanup 2023-03-24 16:01:09 -05:00
api_directive.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
api_directive_internal_test.go Add DAX - full list of squashed commits below 2022-12-12 09:01:20 -08:00
api_directive_test.go Database isolation: Balancer (#2407) 2023-01-19 22:10:08 +00:00
api_test.go Linters! (#2314) 2023-03-10 15:13:15 -06:00
apimethod_string.go Drop the ingest subpackage and related endpoints. 2022-11-15 11:31:10 -08:00
apply.go Sup 294 pre sort command (#2209) 2023-01-23 12:26:38 -06:00
arrow.go fb-1915 Support large id's in NDJSON (#2290) 2023-03-03 15:37:56 -06:00
arrow_test.go [FB-1822] Change dataframe disk format to Arrow from parquet (#2376) 2023-01-10 23:22:58 +00:00
audit.go Updated dependency paths to reflect new repo location 2022-09-06 09:39:22 -07:00
audit_internal_test.go Updated dependency paths to reflect new repo location 2022-09-06 09:39:22 -07:00
audit_test.go fix import paths and import cycles 2023-01-12 00:31:14 +00:00
broadcast.go Dataframe (#2241) 2022-12-12 09:01:20 -08:00
bsi.go Add DAX - full list of squashed commits below 2022-12-12 09:01:20 -08:00
bsi_test.go Add DAX - full list of squashed commits below 2022-12-12 09:01:20 -08:00
cache.go fix go deps, add lattice 2023-01-20 02:11:00 +00:00
cache_test.go linter: prealloc (#2315) 2023-03-11 21:19:05 -06:00
catcher.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
cluster.go linter directive should not have space (dummy commit to trigger CI) (#2228) 2023-01-25 17:00:23 -06:00
cluster_internal_test.go drop anti-entropy feature, since it doesn't work 2022-11-15 11:31:13 -08:00
CODE_OF_CONDUCT.md Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
const_amd64.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
const_other.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
dataframe_test.go string support for dataframe (#2405) 2023-01-19 21:51:16 +00:00
dbshard.go staticcheck fixes (#2278) 2022-11-15 11:33:10 -08:00
dbshard_internal_test.go merge: featurebase merge updates for 2022-10-28 (#2188) 2022-10-28 13:08:23 -04:00
dbshard_test.go cleaned up sync from private repo 2022-09-30 11:20:58 -07:00
delete_test.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
diagnostics.go Updated dependency paths to reflect new repo location 2022-09-06 09:39:22 -07:00
diagnostics_internal_test.go tweak a bunch of logging and config (#2234) 2023-02-03 14:59:07 -06:00
doc.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
docker-compose.yml version update 2023-04-04 08:38:32 -05:00
Dockerfile version update 2023-04-04 08:38:32 -05:00
Dockerfile-clustertests introduce performance counters and system table fanout, plus refactor metrics (#2363) 2023-01-19 21:35:02 +00:00
Dockerfile-clustertests-client introduce performance counters and system table fanout, plus refactor metrics (#2363) 2023-01-19 21:35:02 +00:00
Dockerfile-datagen Add DAX - full list of squashed commits below 2022-12-12 09:01:20 -08:00
Dockerfile-dax move DD profiling/tracing setup into command where it belongs (#2233) 2023-02-01 16:20:39 -06:00
Dockerfile-dax-quick Add DAX - full list of squashed commits below 2022-12-12 09:01:20 -08:00
Dockerfile-fbsql Add kafka support to CLI (fbsql) (#2278) 2023-03-07 08:18:22 -06:00
et_test.go Sync from private repo to commit 12d608c80d 2022-12-12 09:01:20 -08:00
event.go Updated dependency paths to reflect new repo location 2022-09-06 09:39:22 -07:00
executor.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
executor_internal_test.go Dataframe (#2241) 2022-12-12 09:01:20 -08:00
executor_test.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
field.go standardize existence-tracking logic a bit better 2023-03-24 16:01:09 -05:00
field_internal_test.go use separate qcx for write/read in test 2023-01-10 23:22:58 +00:00
field_test.go merge: featurebase merge updates for 2022-10-28 (#2188) 2022-10-28 13:08:23 -04:00
filesystem.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
fragment.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
fragment_internal_test.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
gc.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
gid.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
go.mod Cloud 1358 bolt postgres (#2286) 2023-03-20 09:02:22 -05:00
go.sum Cloud 1358 bolt postgres (#2286) 2023-03-20 09:02:22 -05:00
hack.go Sup 294 pre sort command (#2209) 2023-01-23 12:26:38 -06:00
handler.go Don't use reflect.DeepEqual to compare errors. 2022-12-12 09:01:20 -08:00
handler_test.go Don't use reflect.DeepEqual to compare errors. 2022-12-12 09:01:20 -08:00
holder.go controller commit to DB first then send directives (#2259) 2023-02-17 09:12:15 -06:00
holder_internal_test.go merge: featurebase merge updates for 2022-10-28 (#2188) 2022-10-28 13:08:23 -04:00
holder_test.go Fixed a few more conflicts around imports with the latest sync 2022-11-17 10:00:53 -08:00
http_handler.go linter: prealloc (#2315) 2023-03-11 21:19:05 -06:00
http_handler_internal_test.go fix: updating code to meet linting requirements (#2171) 2022-09-29 12:34:29 -04:00
http_handler_test.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
http_translator.go Enable linter: stylecheck (#2317) 2023-03-14 08:45:18 -05:00
http_translator_test.go merge: featurebase merge updates for 2022-10-28 (#2188) 2022-10-28 13:08:23 -04:00
idalloc.go Linters! (#2314) 2023-03-10 15:13:15 -06:00
idalloc_test.go Updated dependency paths to reflect new repo location 2022-09-06 09:39:22 -07:00
importer.go introduce performance counters and system table fanout, plus refactor metrics (#2363) 2023-01-19 21:35:02 +00:00
index.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
index_internal_test.go merge: featurebase merge updates for 2022-10-28 (#2188) 2022-10-28 13:08:23 -04:00
index_test.go don't use things that instantly exit in a code path tests hit 2023-03-06 13:12:22 -06:00
internal_client.go use primary as source for FieldTranslate backup data (#2255) 2023-02-16 12:13:11 -06:00
internal_client_test.go rework and consolidate view name cleanup 2023-03-24 16:01:09 -05:00
iterator.go Sup 294 pre sort command (#2209) 2023-01-23 12:26:38 -06:00
iterator_internal_test.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
LICENSE update license to full license 2023-01-12 18:23:09 -06:00
LICENSE-2.0.txt Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
license.exceptions Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
like.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
like_test.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
main_test.go use testhook test cleanup 2022-09-30 11:25:27 -07:00
Makefile update ECR-related Makefile targets to get account ID automatically (#2357) 2023-03-30 13:26:09 -05:00
metrics.go Enable linter: stylecheck (#2317) 2023-03-14 08:45:18 -05:00
nfpm.yaml Build target also builds fbsql, fbsql is also packaged. (#2260) 2023-02-21 08:55:27 -08:00
NOTICE Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
null_test.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
performancecounters.go introduce performance counters and system table fanout, plus refactor metrics (#2363) 2023-01-19 21:35:02 +00:00
pilosa.go SUP-302 (#2243) 2023-02-22 11:30:14 -06:00
pilosa_internal_test.go Updated dependency paths to reflect new repo location 2022-09-06 09:39:22 -07:00
pilosa_test.go SUP-302 (#2243) 2023-02-22 11:30:14 -06:00
pprof.go Updated dependency paths to reflect new repo location 2022-09-06 09:39:22 -07:00
rbf.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
README.md version update 2023-04-04 08:38:32 -05:00
row.go introduce performance counters and system table fanout, plus refactor metrics (#2363) 2023-01-19 21:35:02 +00:00
row_test.go fix import paths and import cycles 2023-01-12 00:31:14 +00:00
schema.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
serializer.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
server.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
server_internal_test.go drop anti-entropy feature, since it doesn't work 2022-11-15 11:31:13 -08:00
server_test.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
sql_test.go Sync from private repo to commit 12d608c80d 2022-12-12 09:01:20 -08:00
stattx.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
systemlayer.go implement fb_exec_requests system table (#2327) 2022-12-12 09:01:20 -08:00
time.go staticcheck fixes (#2278) 2022-11-15 11:33:10 -08:00
time_internal_test.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
tracker.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
tracker_test.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
transaction.go Updated dependency paths to reflect new repo location 2022-09-06 09:39:22 -07:00
transaction_test.go Sync from private repo to commit 12d608c80d 2022-12-12 09:01:20 -08:00
translate.go improve ramdisk config in Makefile, use it for everything in tests 2023-02-22 13:19:06 -06:00
translate_boltdb.go fb-1915 Support large id's in NDJSON (#2290) 2023-03-03 15:37:56 -06:00
translate_boltdb_internal_test.go remove in mem translate store 2023-01-10 23:23:18 +00:00
translate_boltdb_test.go resolve go vet errors related to redeclares 2023-01-12 20:32:49 +00:00
translator_test.go resolve go vet errors related to redeclares 2023-01-12 20:32:49 +00:00
tx.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
tx_internal_test.go significant refactor of test setup and teardown 2022-09-30 11:25:27 -07:00
tx_test.go Fixed a few more conflicts around imports with the latest sync 2022-11-17 10:00:53 -08:00
txfactory.go Add DAX - full list of squashed commits below 2022-12-12 09:01:20 -08:00
txfactory_internal_test.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
util.go continue removing Tx parameters to view-type functions 2022-09-30 11:25:27 -07:00
util_test.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
utils_internal_test.go refactor testing to share clusters more often 2022-09-30 11:10:47 -07:00
verchk.go Fb 1975 (#2254) 2023-02-17 13:50:45 -06:00
version.go Updated code to latest version for open-sourcing. 2022-09-02 13:23:39 -07:00
view.go support null/non-null tests for non-BSI fields 2023-03-24 16:01:09 -05:00
view_internal_test.go significant refactor of test setup and teardown 2022-09-30 11:25:27 -07:00
wire_response.go Add kafka support to CLI (fbsql) (#2278) 2023-03-07 08:18:22 -06:00

FeatureBase

This is FeatureBase, a crazy fast database/analytics engine based on B-trees and Roaring Bitmaps.

FeatureBase is written in Go and supports SQL.

License

FeatureBase is released under the Apache 2.0 Open Source license.

Getting Started

You need to have Docker and git installed to use this guide.

Build FeatureBase Server with Docker (Optional)

To build Featurebase from source with Docker, run the following from a prompt:

git clone https://github.com/FeatureBaseDB/featurebase.git
cd featurebase
docker build -t featurebasedb/featurebase:latest .

Run FeatureBase from Docker

To run FeatureBase from Docker, enter the following from a prompt:

docker run -p 10101:10101 featurebasedb/featurebase:latest

Create a Database and Insert Data

To create a sample database, run the following:

curl -X POST http://0.0.0.0:10101/sql -d 'CREATE TABLE planets (_id id, planet_name string, moons stringset, diameter_in_km int);'

To insert data into the planets database, do the following:

curl -X POST http://0.0.0.0:10101/sql -d "INSERT INTO planets VALUES (1, 'Mercury', [''], 4879), (2, 'Venus', [''], 12104), (3, 'Earth', ['Moon'], 12742);" 

You may insert as many records as you like in a single POST.

Query Data

To query the data in a FeatureBase database, do the following:

curl -X POST http://0.0.0.0:10101/sql -d "SELECT * FROM planets;" | python3 -m json.tool

The output should be as follows:

{
    "schema": {
        "fields": [
            {
                "name": "_id",
                "type": "id",
                "base-type": "id",
                "type-info": null
            },
            {
                "name": "planet_name",
                "type": "string",
                "base-type": "string",
                "type-info": null
            },
            {
                "name": "moons",
                "type": "stringset",
                "base-type": "stringset",
                "type-info": null
            },
            {
                "name": "diameter_in_km",
                "type": "int",
                "base-type": "int",
                "type-info": null
            }
        ]
    },
    "data": [
        [
            1,
            "Mercury",
            null,
            4879
        ],
        [
            2,
            "Venus",
            null,
            12104
        ],
        [
            3,
            "Earth",
            [
                "Moon"
            ],
            12742
        ]
    ],
    "execution-time": 1006
}

Community and Support

Join us on Discord.