scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-08-28 10:54:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

BSI data model

Browse source
This commit is contained in:
Michael Baird 2017-09-27 16:53:02 -05:00
parent 35d6e945a8
commit d2c052d4a9
1 changed files with 20 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

20
docs/data-model.md
View file

@ -101,3 +101,23 @@ SetBit(frame="A", rowID=8, columnID=3, timestamp="2017-05-19T00:00")
```
![time quantum frame diagram](/img/docs/frame-time-quantum.svg)
#### BSI Range-Encoding
Bit-Sliced Indexing (BSI) is the storage method Pilosa uses to represent multi-bit integers in a bitmap index. Integers are stored as n-bit, range-encoded
bit-sliced indexes of base-2, along with an additional bitmap indicating "not null". This means that a 16-bit integer will require 17 bitmaps: one for each 0-bit of the 16 bit-slice components (the 1-bit does not need to be stored because with range-encoding the highest bit position is always 1) and one for the non-null bitmap. Pilosa can evaluate, aggregate, and range queries on these BSI integers.
Internally Pilosa stores each BSI `field` as a `view` within a `frame`. The 'rowIDs' of the `view` are composed of the base-2 representation of the integer. Pilosa manages the base-2 offset and translation that efficiently packs the integer value within the minimum set of rows.
For example, the following `SetFieldValue()` queries will result in the data described in the illustration below:
```
SetFieldValue(col=1, frame="A", "field0"=1)
SetFieldValue(col=2, frame="A", "field0"=2)
SetFieldValue(col=3, frame="A", "field0"=3)
SetFieldValue(col=4, frame="A", "field0"=7)
SetFieldValue(col=2, frame="A", "field1"=1)
SetFieldValue(col=3, frame="A", "field1"=6)
```
![BSI diagram](/img/docs/frame-bsi.svg)