scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-10-07 03:17:50 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Updated Pilosa Schema table and added pdk repo link

Browse source
This commit is contained in:
Ashley Svetlik 2019-07-24 09:32:18 -05:00
parent bd00f1bfe2
commit 6d58db4fc9
1 changed files with 12 additions and 11 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

23
docs/pdk.md
View file

@ -38,17 +38,17 @@ the record to arrive at that field. For example:
This JSON object would result in the following Pilosa schema:
| Field | Type | Min | Max | Size |
|----------------|--------|-----|------------|--------|
| name | ranked | | | 100000 |
| favorite_foods | ranked | | | 100000 |
| default | ranked | | | 100000 |
| age | int | 0 | 2147483647 | |
| location | ranked | | | 1000 |
| latitude | int | 0 | 2147483647 | |
| longitude | int | 0 | 2147483647 | |
| location-city | ranked | | | 100000 |
| location-state | ranked | | | 100000 |
| Field | Value | Type | Cache Size |
|----------------|------------------------------|--------|------------|
| name | "jill" | ranked | 100000 |
| favorite_foods | "corn chips", "chipotle dip" | ranked | 100000 |
| default | | ranked | 100000 |
| age | 27 | int | |
| location | | ranked | 1000 |
| latitude | 3754 | int | |
| longitude | 4526 | int | |
| location-city | "Austin" | ranked | 100000 |
| location-state | "Texas" | ranked | 100000 |
All set fields are created as ranked fields by default, with the cache size
listed above. Integer fields are created with a minimum size of zero and a
@ -66,6 +66,7 @@ control over the way data is indexed, and ingestion performance.
* `--subject-path`: If nothing is passed for this option, then each record will be assigned a unique sequential column ID. If `subject-path` is specified, then the value at this path in the record will be mapped to a column ID. If the same value appears in another record, the same column ID will be used.
* `--proxy`: The PDK ingests data, but also keeps a mapping for string values to row IDs, and from subjects to column ids. Because of this, querying Pilosa directly may not be useful, since it only returns integer row and column ids. The PDK will start a proxy server which intercepts requests to Pilosa using strings for row and column ids, and translates them to the integers that Pilosa understands. It will also translate responses so that (e.g.) a TopN query will return `{"results":[[{"Key":"chipotle dip","Count":1},{"Key":"corn chips","Count":1}]]}`. By default, the mapping is stored in an embedded leveldb.
For more information on running `pdk kafka` and how Pilosa interfaces with Kafka, please see the [pdk kafka repository](https://github.com/pilosa/pdk/tree/master/kafka)
### Library