mirror of
https://github.com/featurebasedb/featurebase.git
synced 2026-09-07 17:15:56 +00:00
Address review comments
This commit is contained in:
parent
f0a4e5ab82
commit
4fcf5ef1ca
4 changed files with 7 additions and 7 deletions
|
|
@ -1,6 +1,6 @@
|
|||
# Contributing to Pilosa
|
||||
|
||||
The workflow components of these instructions apply to all of our repos.
|
||||
The workflow components of these instructions apply to all Pilosa repositories.
|
||||
|
||||
## Reporting a bug
|
||||
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ nav = [
|
|||
|
||||
## Client Libraries
|
||||
|
||||
This section contains example code for client libraries in several languages. Please remember that when modeling your data in Pilosa, it is best to keep row and column ids sequential. It is best to avoid using the the output of a hash, or randomly distributed ids with Pilosa.
|
||||
This section contains example code for client libraries in several languages. Please remember that when modeling your data in Pilosa, it is best to keep row and column ids sequential. It is best to avoid using the output of a hash or randomly distributed ids with Pilosa.
|
||||
|
||||
### Go
|
||||
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ The central component of Pilosa's data model is a boolean matrix. Each cell in t
|
|||
|
||||
Rows and columns can represent anything (they could even represent the same set of things - a [bigraph](https://en.wikipedia.org/wiki/Bigraph)). Pilosa can associate arbitrary key/value pairs (referred to as attributes) to rows and columns, but queries and storage are optimized around the core matrix.
|
||||
|
||||
Pilosa lays out data first in rows, so queries which get all the set bits in one or many rows, or compute a combining operation on multiple rows such as Intersect or Union are the fastest. Pilosa categorizes rows into different *frames* and quickly retrieve the top rows in a frame sorted by the number of bits set in each row.
|
||||
Pilosa lays out data first in rows, so queries which get all the set bits in one or many rows, or compute a combining operation on multiple rows such as Intersect or Union are the fastest. Pilosa categorizes rows into different *frames* and quickly retrieves the top rows in a frame sorted by the number of bits set in each row.
|
||||
|
||||
Please note that Pilosa is most performant when row and column IDs are sequential starting from 0. You can deviate from this to some degree, but setting a bit with column ID 2<sup>63</sup> on a single-node cluster, for example, will not work well due to memory limitations.
|
||||
|
||||
|
|
@ -43,11 +43,11 @@ Row ids are sequential increasing integers namespaced to each Frame within an In
|
|||
|
||||
### Frame
|
||||
|
||||
Frames are used to segment rows within an index, for example to defining different functional groups. A frame might correspond to a single field in a relational table, where each row in a standard frame represents a single possible value of the field. Similarly, a frame with BSI values could represent all possible integer values of a field .
|
||||
Frames are used to segment rows within an index, for example to define different functional groups. A frame might correspond to a single field in a relational table, where each row in a standard frame represents a single possible value of the field. Similarly, a frame with BSI values could represent all possible integer values of a field .
|
||||
|
||||
#### Relational Analogy
|
||||
|
||||
The Pilosa index is a flexible structure; it can represent any sort of high-cardinality binary matrix. The most common pattern we have encountered in Pilosa use cases is a direct analogy to the relational model, summarized here.
|
||||
The Pilosa index is a flexible structure; it can represent any sort of high-cardinality binary matrix. We have explored a number of modeling patterns in Pilosa use cases; one accessible example is a direct analogy to the relational model, summarized here.
|
||||
|
||||
Entities:
|
||||
|
||||
|
|
@ -137,7 +137,7 @@ SetBit(frame="A", row=8, col=3, timestamp="2017-05-19T00:00")
|
|||
|
||||
#### BSI Range-Encoding
|
||||
|
||||
Bit-Sliced Indexing (BSI) is the storage method Pilosa uses to represent multi-bit integers in a bitmap index. Integers are stored as n-bit, range-encoded bit-sliced indexes of base-2, along with an additional bitmap indicating "not null". This means that a 16-bit integer will require 17 bitmaps: one for each 0-bit of the 16 bit-slice components (the 1-bit does not need to be stored because with range-encoding the highest bit position is always 1) and one for the non-null bitmap. Pilosa can evaluate `Range`, `Min`, `Max`, and `Sum` queries on these BSI integers.
|
||||
Bit-Sliced Indexing (BSI) is the storage method Pilosa uses to represent multi-bit integers in a bitmap index. Integers are stored as n-bit, range-encoded bit-sliced indexes of base-2, along with an additional bitmap indicating "not null". This means that a 16-bit integer will require 17 bitmaps: one for each 0-bit of the 16 bit-slice components (the 1-bit does not need to be stored because with range-encoding the highest bit position is always 1) and one for the non-null bitmap. Pilosa can evaluate `Range`, `Min`, `Max`, and `Sum` queries on these BSI integers. The result of a `Sum` query includes a count, which can be used to compute an average with no other overhead.
|
||||
|
||||
Internally Pilosa stores each BSI `field` as a `view` within a `frame`. The rows of the `view` contain the base-2 representations of the integer values. Pilosa manages the base-2 offset and translation that efficiently packs the integer value within the minimum set of rows.
|
||||
|
||||
|
|
|
|||
|
|
@ -204,7 +204,7 @@ for pcount, topn in zip(pcounts, resp.json()['results']):
|
|||
```
|
||||
|
||||
<div class="note">
|
||||
Note that the <a href="../data-model/#bsi-range-encoding">BSI</a>-powered <a href="../query-language/#sum">Sum</a> query now provides a an alternative approach to this kind of query.
|
||||
Note that the <a href="../data-model/#bsi-range-encoding">BSI</a>-powered <a href="../query-language/#sum">Sum</a> query now provides an alternative approach to this kind of query.
|
||||
</div>
|
||||
|
||||
For more examples and details, see this [ipython notebook](https://github.com/pilosa/notebooks/blob/master/taxi-use-case.ipynb).
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue