scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-09-10 23:11:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge pull request #1224 from travisturner/docs-row-col

Browse source 
remove references to row and column labels from the docs
This commit is contained in:
Travis Turner 2018-04-23 18:56:00 -05:00 • committed by GitHub
commit 579234d216
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
6 changed files with 46 additions and 46 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
docs/administration.md
View file

@ -48,9 +48,9 @@ On Mac OS X, `ulimit` does not behave predictably. [This blog post](https://blog
#### Importing
The import API expects a csv of RowID,ColumnID's.
The import API expects a csv of rowID,columnID's.
When importing large datasets remember it is much faster to pre sort the data by RowID and then by ColumnID in ascending order. You can use the `--sort` flag to do that. Also, avoid querying Pilosa until the import is complete, otherwise you will experience inconsistent results.
When importing large datasets remember it is much faster to pre sort the data by row ID and then by column ID in ascending order. You can use the `--sort` flag to do that. Also, avoid querying Pilosa until the import is complete, otherwise you will experience inconsistent results.
```
pilosa import --sort -i project -f stargazer project-stargazer.csv
@ -70,7 +70,7 @@ pilosa import -i project -f stargazer --field star_count project-stargazer-count
#### Exporting
Exporting Data to csv can be performed on a live instance of Pilosa. You need to specify the Index, Frame, and View(default is standard). The API also expects the slice number, but the `pilosa export` sub command will export all slices within a Frame. The data will be in csv format RowID,ColumnID and sorted by column ID.
Exporting Data to csv can be performed on a live instance of Pilosa. You need to specify the Index, Frame, and View(default is standard). The API also expects the slice number, but the `pilosa export` sub command will export all slices within a Frame. The data will be in csv format rowID,columnID and sorted by columnID.
```
curl "http://localhost:10101/export?index=repository&frame=stargazer&slice=0&view=standard" \
--header "Accept: text/csv"

4
docs/api-reference.md
View file

@ -68,7 +68,7 @@ Sends a query to the Pilosa server with the given index. The request body is UTF
``` request
curl localhost:10101/index/user/query \
-X POST \
-d 'Bitmap(frame="language", rowID=5)'
-d 'Bitmap(frame="language", row=5)'
```
``` response
{"results":[{"attrs":{},"bits":[100]}]}
@ -83,7 +83,7 @@ The query is executed for all [slices](../data-model/#slice) by default. To use
``` request
curl "localhost:10101/index/user/query?columnAttrs=true&slices=0,1" \
-X POST \
-d 'Bitmap(frame="language", rowID=5)'
-d 'Bitmap(frame="language", row=5)'
```
``` response
{

18
docs/data-model.md
View file

@ -88,8 +88,8 @@ The standard View contains the same Row/Column format as the input data.
If a Frame has a time quantum, then Views are generated for each of the defined time segments. For example, for a frame with a time quantum of `YMD`, the following `SetBit()` queries will result in the data described in the illustration below:
```
SetBit(frame="A", rowID=8, columnID=3, timestamp="2017-05-18T00:00")
SetBit(frame="A", rowID=8, columnID=3, timestamp="2017-05-19T00:00")
SetBit(frame="A", row=8, col=3, timestamp="2017-05-18T00:00")
SetBit(frame="A", row=8, col=3, timestamp="2017-05-19T00:00")
```
![time quantum frame diagram](/img/docs/frame-time-quantum.svg)
@ -100,17 +100,17 @@ SetBit(frame="A", rowID=8, columnID=3, timestamp="2017-05-19T00:00")
Bit-Sliced Indexing (BSI) is the storage method Pilosa uses to represent multi-bit integers in a bitmap index. Integers are stored as n-bit, range-encoded
bit-sliced indexes of base-2, along with an additional bitmap indicating "not null". This means that a 16-bit integer will require 17 bitmaps: one for each 0-bit of the 16 bit-slice components (the 1-bit does not need to be stored because with range-encoding the highest bit position is always 1) and one for the non-null bitmap. Pilosa can evaluate `Sum` and `Range` queries on these BSI integers.
Internally Pilosa stores each BSI `field` as a `view` within a `frame`. The 'rowIDs' of the `view` are composed of the base-2 representation of the integer. Pilosa manages the base-2 offset and translation that efficiently packs the integer value within the minimum set of rows.
Internally Pilosa stores each BSI `field` as a `view` within a `frame`. The rows of the `view` are composed of the base-2 representation of the integer. Pilosa manages the base-2 offset and translation that efficiently packs the integer value within the minimum set of rows.
For example, the following `SetFieldValue()` queries will result in the data described in the illustration below:
```
SetFieldValue(columnID=1, frame="A", field0=1)
SetFieldValue(columnID=2, frame="A", field0=2)
SetFieldValue(columnID=3, frame="A", field0=3)
SetFieldValue(columnID=4, frame="A", field0=7)
SetFieldValue(columnID=2, frame="A", field1=1)
SetFieldValue(columnID=3, frame="A", field1=6)
SetFieldValue(col=1, frame="A", field0=1)
SetFieldValue(col=2, frame="A", field0=2)
SetFieldValue(col=3, frame="A", field0=3)
SetFieldValue(col=4, frame="A", field0=7)
SetFieldValue(col=2, frame="A", field1=1)
SetFieldValue(col=3, frame="A", field1=6)
```
![BSI frame diagram](/img/docs/frame-bsi.svg)

4
docs/glossary.md
View file

@ -24,7 +24,7 @@ nav = []
<strong id="fragment">Fragment:</strong> A Fragment is the intersection of a [frame](#frame) and a [slice](#slice) in an [index](#index).
<strong id="frame">[Frame](../data-model/#frame):</strong> Frames are used to group [rows](#row) into different categories. `RowID`s are namespaced by frame such that the same `RowID` in a different frame refers to a different row. For [ranked](#topn) frames, rows are kept in sorted order within the frame.
<strong id="frame">[Frame](../data-model/#frame):</strong> Frames are used to group [rows](#row) into different categories. Row IDs are namespaced by frame such that the same row ID in a different frame refers to a different row. For [ranked](#topn) frames, rows are kept in sorted order within the frame.
<strong id="index">[Index](../data-model/#index):</strong> An Index is a top level container in Pilosa, analogous to a database in an RDBMS. Queries cannot operate across multiple indexes.
@ -62,6 +62,6 @@ nav = []
<strong id="toml">[TOML](https://github.com/toml-lang/toml):</strong> the language used for Pilosa's [configuration file](../configuration/).
<strong id="topn">[TopN](../query-language/#topn):</strong> A [PQL](#pql) query that returns a list of `RowID`s, sorted by the count of [bits](#bit) set in the [row](#row), within a specified [frame](#frame).
<strong id="topn">[TopN](../query-language/#topn):</strong> A [PQL](#pql) query that returns a list of row IDs, sorted by the count of [bits](#bit) set in the [row](#row), within a specified [frame](#frame).
<strong id="view">[View](../data-model/#view):</strong> Views separate the different data layouts within a [Frame](#frame). The primary view is standard, which represents the typical [row](#row)/[column](#column) data. Time based frame views are automatically generated for each [time quantum](#time-quantum). Views are internally managed by Pilosa, and never exposed directly via the API. This simplifies the functional interface by separating it from the physical data representation.

2
docs/pdk.md
View file

@ -34,7 +34,7 @@ With this definition available, the PDK tool can run the import, which consists
- for each CSV record:
- generate a columnID
- apply all ParserMappers, generating a list of (frame, ID) pairs
- set the appropriate bit. schematically: SetBit(id=rowID, frame=frame, profileID=columnID)
- set the appropriate bit. schematically: SetBit(row=rowID, frame=frame, col=columnID)
The process is summarized in this flowchart:

58
docs/query-language.md
View file

@ -25,17 +25,17 @@ There will be one item in the `results` array for each PQL query in the request.
* Angle Brackets `<>` denote required arguments
* Square Brackets `[]` denote optional arguments
* UPPER_CASE denotes a descriptor that will need to be filled in with a concrete value (e.g. `ROW_LABEL`, `STRING`)
* UPPER_CASE denotes a descriptor that will need to be filled in with a concrete value (e.g. `ATTR_NAME`, `STRING`)
##### Examples
Before running any of the example queries below, follow the instructions in the [Getting Started](../getting-started/) section to set up an index, frames, and populate them with some data.
The examples just show the PQL quer(ies) needed - to run the query `SetBit(frame="stargazer", columnID=10, rowID=1)` against a server using curl, you would:
The examples just show the PQL quer(ies) needed - to run the query `SetBit(frame="stargazer", col=10, row=1)` against a server using curl, you would:
``` request
curl localhost:10101/index/repository/query \
-X POST \
-d 'SetBit(frame="stargazer", columnID=10, rowID=1)'
-d 'SetBit(frame="stargazer", col=10, row=1)'
```
``` response
{"results":[true]}
@ -58,7 +58,7 @@ curl localhost:10101/index/repository/query \
**Spec:**
```
SetBit(<frame=STRING>, <ROW_LABEL=UINT>, <COL_LABEL=UINT>,
SetBit(<frame=STRING>, <row=UINT>, <col=UINT>,
[timestamp=TIMESTAMP])
```
@ -76,26 +76,26 @@ A return value of `false` indicates that the bit was already set to 1 and nothin
**Examples:**
```
SetBit(frame="stargazer", columnID=10, rowID=1)
SetBit(frame="stargazer", col=10, row=1)
```
This query illustrates setting a bit in the stargazer frame. User with id=1 has starred repository with id=10.
SetBit also supports providing a timestamp. To write the date that a user starred a repository.
```
SetBit(frame="stargazer", columnID=10, rowID=1, timestamp="2016-01-01T00:00")
SetBit(frame="stargazer", col=10, row=1, timestamp="2016-01-01T00:00")
```
Setting multiple bits in a single request:
```
SetBit(frame="stargazer", columnID=10, rowID=1) SetBit(frame="stargazer", columnID=10, rowID=2) SetBit(frame="stargazer", columnID=20, rowID=1) SetBit(frame="stargazer", columnID=30, rowID=2)
SetBit(frame="stargazer", col=10, row=1) SetBit(frame="stargazer", col=10, row=2) SetBit(frame="stargazer", col=20, row=1) SetBit(frame="stargazer", col=30, row=2)
```
#### SetRowAttrs
**Spec:**
```
SetRowAttrs(<frame=STRING>, <ROW_LABEL=UINT>,
SetRowAttrs(<frame=STRING>, <row=UINT>,
<ATTR_NAME=ATTR_VALUE>,
[ATTR_NAME=ATTR_VALUE ...])
```
@ -111,13 +111,13 @@ SetRowAttrs queries always return `null` upon success.
**Examples:**
```
SetRowAttrs(frame="stargazer", rowID=10, username="mrpi", active=true)
SetRowAttrs(frame="stargazer", row=10, username="mrpi", active=true)
```
Set username value and active status for user 10. These are arbitrary key/value pairs which have no meaning to Pilosa. You can see the attributes you've set on a row with a [Bitmap](../query-language/#bitmap) query like so `Bitmap(frame="stargazer", stargazer_id=10)`.
```
SetRowAttrs(frame="stargazer", rowID=10, username=null)
SetRowAttrs(frame="stargazer", row=10, username=null)
```
Delete username value for user 10.
@ -127,7 +127,7 @@ Delete username value for user 10.
**Spec:**
```
SetColumnAttrs(<frame=STRING>, <ROW_LABEL=UINT>,
SetColumnAttrs(<frame=STRING>, <row=UINT>,
<ATTR_NAME=ATTR_VALUE>,
[ATTR_NAME=ATTR_VALUE ...])
```
@ -143,13 +143,13 @@ SetColumnAttrs queries always return `null` upon success. Setting a value of `nu
**Examples:**
```
SetColumnAttrs(columnID=10, stars=123, url="http://projects.pilosa.com/10", active=true)
SetColumnAttrs(col=10, stars=123, url="http://projects.pilosa.com/10", active=true)
```
Set url value and active status for project 10. These are arbitrary key/value pairs which have no meaning to Pilosa. You can see the attributes you've set on a column with a [Bitmap](../query-language/#bitmap) query like so `Bitmap(frame="stargazer", columnID=10)`.
Set url value and active status for project 10. These are arbitrary key/value pairs which have no meaning to Pilosa. You can see the attributes you've set on a column with a [Bitmap](../query-language/#bitmap) query like so `Bitmap(frame="stargazer", col=10)`.
```
SetColumnAttrs(columnID=10, url=null)
SetColumnAttrs(col=10, url=null)
```
Delete url value for repo 10.
@ -160,7 +160,7 @@ Delete url value for repo 10.
**Spec:**
```
SetBit(<frame=STRING>, <ROW_LABEL=UINT>, <COL_LABEL=UINT>,
SetBit(<frame=STRING>, <row=UINT>, <col=UINT>,
[timestamp=TIMESTAMP])
```
@ -177,7 +177,7 @@ A return value of `false` indicates that the bit was already set to 0 and nothin
**Examples:**
```
ClearBit(frame="stargazer", columnID=10, rowID=1)
ClearBit(frame="stargazer", col=10, row=1)
```
Remove relationship between the stargazer in row 1 and the repository in column 10 from the stargazer frame.
@ -188,12 +188,12 @@ Remove relationship between the stargazer in row 1 and the repository in column
**Spec:**
```
SetFieldValue(<COL_LABEL=UINT>, <frame=STRING>, <FIELD_NAME=INT>)
SetFieldValue(<col=UINT>, <frame=STRING>, <FIELD_NAME=INT>)
```
**Description:**
`SetFieldValue` assigns an integer value with the specified field name to the `columnID` in the given `frame`.
`SetFieldValue` assigns an integer value with the specified field name to the `col` in the given `frame`.
**Result Type:** null
@ -203,7 +203,7 @@ SetFieldValue returns `null` upon success.
Set the number of pull requests of repository 10.
```
SetFieldValue(columnID=10, frame="stats", pullrequests=2)
SetFieldValue(col=10, frame="stats", pullrequests=2)
```
@ -214,7 +214,7 @@ SetFieldValue(columnID=10, frame="stats", pullrequests=2)
**Spec:**
```
Bitmap(<frame=STRING>, (<ROW_LABEL=UINT> | <COL_LABEL>=UINT))
Bitmap(<frame=STRING>, (<rowL=UINT> | <col>=UINT))
```
**Description:**
@ -229,7 +229,7 @@ e.g. `{"attrs":{"username":"mrpi","active":true},"bits":[10, 20]}`
Query all repositories that user 1 has starred.
```
Bitmap(frame="stargazer", rowID=1)
Bitmap(frame="stargazer", row=1)
```
Returns `{"attrs":{"username":"mrpi","active":true},"bits":[10, 20]}`
@ -286,7 +286,7 @@ attrs will always be empty
Query repositories which have been starred by two users.
```
Intersect(Bitmap(frame="stargazer", rowID=1), Bitmap(frame="stargazer", rowID=2))
Intersect(Bitmap(frame="stargazer", row=1), Bitmap(frame="stargazer", row=2))
```
Returns `{"attrs":{},"bits":[10]}`.
@ -313,7 +313,7 @@ attrs will always be empty
Query repositories which have been starred by one user and not another.
```
Difference(Bitmap(frame="stargazer", rowID=1), Bitmap( frame="stargazer", rowID=2))
Difference(Bitmap(frame="stargazer", row=1), Bitmap( frame="stargazer", row=2))
```
Return `{"results":[{"attrs":{},"bits":[20]}]}`
@ -321,7 +321,7 @@ Return `{"results":[{"attrs":{},"bits":[20]}]}`
* bits are repositories that were starred by user 1 BUT NOT user 2
```
Difference(Bitmap(frame="stargazer", rowID=2), Bitmap( frame="stargazer", rowID=1))
Difference(Bitmap(frame="stargazer", row=2), Bitmap( frame="stargazer", row=1))
```
Return `{"attrs":{},"bits":[30]}`
@ -349,7 +349,7 @@ attrs will always be empty
Query repositories which have been starred by two users.
```
Xor(Bitmap(frame="stargazer", rowID=1), Bitmap(frame="stargazer", rowID=2))
Xor(Bitmap(frame="stargazer", row=1), Bitmap(frame="stargazer", row=2))
```
Returns `{"attrs":{},"bits":[30]}`.
@ -373,7 +373,7 @@ Returns the number of set bits in the `BITMAP_CALL` passed in.
Query the number of repositories to which a user has contributed.
```
Count(Bitmap(frame="stargazer", rowID=1))
Count(Bitmap(frame="stargazer", row=1))
```
Return `2`
@ -427,7 +427,7 @@ Returns `[{"key": 1, "count": 2}, {"key": 2, "count": 2}]`
* Results are the top two users sorted by number of repositories they've starred in descending order.
```
TopN(Bitmap(frame="language", rowID=1), frame="stargazer", n=2)
TopN(Bitmap(frame="language", row=1), frame="stargazer", n=2)
```
Returns `[{"key": 1, "count": 2}, {"key": 2, "count": 1}]`
@ -439,7 +439,7 @@ Returns `[{"key": 1, "count": 2}, {"key": 2, "count": 1}]`
**Spec:**
```
Range(<frame=STRING>, <ROW_LABEL=UINT>,
Range(<frame=STRING>, <row=UINT>,
<start=TIMESTAMP>, <end=TIMESTAMP>)
```
@ -455,7 +455,7 @@ between the given `start` and `end` timestamps.
When you set timestamp using SetBit, you will able to query all repositories that a user has starred within a date range.
```
Range(frame="stargazer", rowID=1, start="2010-01-01T00:00", end="2017-03-02T03:00")
Range(frame="stargazer", row=1, start="2010-01-01T00:00", end="2017-03-02T03:00")
```
Returns `{{"attrs":{},"bits":[10]}`