mirror of
https://github.com/featurebasedb/featurebase.git
synced 2026-08-28 02:44:59 +00:00
Change the README to be an index of links to the Docs
This commit is contained in:
Travis
parent
03aa9447a8
commit
13b07b99b5
1 changed files with 41 additions and 306 deletions
347
README.md
347
README.md
|
|
@ -1,326 +1,61 @@
|
|||
# pilosa
|
||||
|
||||
Pilosa is a bitmap index.
|
||||
<a href="https://www.pilosa.com">
|
||||
<img src="https://www.pilosa.com/img/logo.svg" width="50%">
|
||||
</a>
|
||||
|
||||
[](https://travis-ci.com/pilosa/pilosa)
|
||||
|
||||
## An open source, distributed bitmap index.
|
||||
- [Docs](#docs)
|
||||
- [Getting Started](#getting-started)
|
||||
- [Data Model](#data-model)
|
||||
- [Client Libraries](#client-drivers)
|
||||
- [Get Support](#get-support)
|
||||
- [Contributing](#contributing)
|
||||
|
||||
|
||||
## Docs
|
||||
|
||||
See our [Documentation](https://www.pilosa.com/docs/) for information about installing and working with Pilosa.
|
||||
|
||||
|
||||
## Getting Started
|
||||
|
||||
Pilosa requires Go 1.7 or greater.
|
||||
[Getting Started](https://www.pilosa.com/docs/getting-started/)
|
||||
|
||||
You can download the source by running `go get`:
|
||||
|
||||
```sh
|
||||
$ go get github.com/pilosa/pilosa
|
||||
```
|
||||
1. [Install Pilosa](https://www.pilosa.com/docs/installation/).
|
||||
|
||||
Now you can install the `pilosa` binary:
|
||||
2. [Start Pilosa](https://www.pilosa.com/docs/getting-started/#starting-pilosa) with the default configuration:
|
||||
|
||||
```sh
|
||||
$ go install github.com/pilosa/pilosa/cmd/...
|
||||
```
|
||||
```shell
|
||||
pilosa server
|
||||
```
|
||||
|
||||
and verify that it's running:
|
||||
|
||||
```shell
|
||||
curl localhost:10101/nodes
|
||||
```
|
||||
|
||||
Now run a single pilosa node with the default configuration:
|
||||
3. Follow along with the [Sample Project](https://www.pilosa.com/docs/getting-started/#sample-project) to get a better understanding of Pilosa's capabilities.
|
||||
|
||||
```sh
|
||||
pilosa server
|
||||
```
|
||||
|
||||
## Configuration
|
||||
## Data Model
|
||||
|
||||
Running just `pilosa` will show a list of available subcommands. `pilosa help
|
||||
<command>` will show usage information and the available flags for the command.
|
||||
Check out how the Pilosa [Data Model](https://www.pilosa.com/docs/data-model/) works.
|
||||
|
||||
Any flag can be specified at the command line, in an environment variables,
|
||||
and/or in a toml config file. The environment variable for any flag is that
|
||||
flag, upper cased, prefixed with `PILOSA_`, and with any dashes converted to
|
||||
underscores. For example, if you would specify `--cluster.poll-interval=30s` at
|
||||
the command line, you would set `PILOSA_CLUSTER.POLL_INTERVAL=30s` in the
|
||||
environment. For the configuration file, a dot in a flag denotes nesting with in
|
||||
the config file. See the example config file below for examples of this.
|
||||
|
||||
You can specify a configuration by setting the `--config` flag when running
|
||||
`pilosa`.
|
||||
## Client Libraries
|
||||
|
||||
There are supported libraries for the following languages:
|
||||
- [Go](https://www.pilosa.com/docs/client-libraries/#go)
|
||||
- [Java](https://www.pilosa.com/docs/client-libraries/#java)
|
||||
- [Python](https://www.pilosa.com/docs/client-libraries/#python)
|
||||
|
||||
```sh
|
||||
pilosa server --config custom-config-file.cfg
|
||||
```
|
||||
## Get Support
|
||||
|
||||
The config file uses the [TOML](https://github.com/toml-lang/toml) configuration file format,
|
||||
and should look like:
|
||||
There are [several channels](https://www.pilosa.com/community/#support) availble for you to reach out to us for support.
|
||||
|
||||
```
|
||||
data-dir = "/tmp/pil0"
|
||||
bind = "127.0.0.1:10101"
|
||||
## Contributing
|
||||
|
||||
[cluster]
|
||||
poll-interval = "2m0s"
|
||||
replicas = 2
|
||||
hosts = [
|
||||
"127.0.0.1:10101",
|
||||
"127.0.0.1:10102",
|
||||
]
|
||||
|
||||
[anti-entropy]
|
||||
interval = "10m0s"
|
||||
|
||||
[profile]
|
||||
cpu = "/home/mycpuprofile"
|
||||
cpu-time = "30s"
|
||||
```
|
||||
|
||||
You can generate a template config file with default values with:
|
||||
|
||||
```sh
|
||||
pilosa config
|
||||
```
|
||||
|
||||
The first two configuration options will be unique to each node in the cluster:
|
||||
|
||||
`data-dir`: directory in which data is stored to disk
|
||||
|
||||
`bind`: IP and port that pilosa will listen on
|
||||
|
||||
The remaining configuration options should be the same on every node in the cluster.
|
||||
|
||||
`[cluster] replicas`: the number of replicas within the cluster
|
||||
|
||||
`[cluster] hosts`: specifies each node within the cluster
|
||||
|
||||
`[cluster] poll-interval`: TODO
|
||||
|
||||
`[anti-entropy] interval`: TODO
|
||||
|
||||
There are also some profiling options for debugging and performance tuning - these don't need to be the same across the cluster and are mostly useful for doing Pilosa development.
|
||||
|
||||
`[profile] cpu`: Path at which to store cpu profiling data which will be taken when pilosa starts.
|
||||
|
||||
`[profile] cpu-time`: Amount of time for which to collect cpu profiling data at startup.
|
||||
|
||||
## Docker
|
||||
|
||||
You can create a Pilosa container using `make docker` or equivalently:
|
||||
```
|
||||
docker build -t pilosa:latest .
|
||||
```
|
||||
|
||||
You can run a temporary container using:
|
||||
```
|
||||
docker run -it --rm --name pilosa -p 10101:10101 pilosa:latest
|
||||
```
|
||||
|
||||
When you click `Ctrl+C` to stop the container, the container and the data in the container will be erased. You can leave out `--rm` flag to keep the data in the container. See [Docker documentation](https://docs.docker.com) for other options.
|
||||
|
||||
## Usage
|
||||
|
||||
You can interact with Pilosa via HTTP requests to the host:port on which you have Pilosa running.
|
||||
The following examples illustrate how to do this using `curl` with a Pilosa cluster running on
|
||||
127.0.0.1 port 10101.
|
||||
|
||||
Return the version of Pilosa:
|
||||
```sh
|
||||
$ curl "http://127.0.0.1:10101/version"
|
||||
```
|
||||
|
||||
Return a list of all indexes and frames in the index:
|
||||
```sh
|
||||
$ curl "http://127.0.0.1:10101/schema"
|
||||
```
|
||||
|
||||
### Index and Frame Schema
|
||||
|
||||
Before running a query, the corresponding index and frame must be created. Note that index and frame names can contain only lower case letters, numbers, dash (`-`), underscore (`_`) and dot (`.`).
|
||||
|
||||
You can create the index `sample-idx` using:
|
||||
|
||||
```sh
|
||||
$ curl -XPOST "http://127.0.0.1:10101/index" \
|
||||
-d '{"index": "sample-idx"}'
|
||||
```
|
||||
|
||||
Optionally, you can specify the column label on index creation:
|
||||
|
||||
```sh
|
||||
$ curl -XPOST "http://127.0.0.1:10101/index" \
|
||||
-d '{"index": "sample-idx", "options": {"columnLabel": "user"}}'
|
||||
```
|
||||
|
||||
The frame `collaboration` may be created using the following call:
|
||||
|
||||
```sh
|
||||
$ curl -XPOST "http://127.0.0.1:10101/frame" \
|
||||
-d '{"index": "sample-idx", "frame": "collaboration"}'
|
||||
```
|
||||
|
||||
It is possible to specify the frame row label on frame creation:
|
||||
|
||||
```sh
|
||||
$ curl -XPOST "http://127.0.0.1:10101/frame" \
|
||||
-d '{"index": "sample-idx", "frame": "collaboration", "options": {"rowLabel": "project"}}'
|
||||
```
|
||||
|
||||
### Queries
|
||||
|
||||
Queries to Pilosa require sending a POST request where the query itself is sent as POST data.
|
||||
You specify the index on which to perform the query with a URL argument `index=index-name`.
|
||||
|
||||
In this section, we assume both the index `sample-idx` with column label `user` and the frame `collaboration` with row label `project` was created.
|
||||
|
||||
A query sent to index `sample-idx` will have the following format:
|
||||
|
||||
```sh
|
||||
$ curl -X POST "http://127.0.0.1:10101/query?index=sample-idx" -d 'Query()'
|
||||
```
|
||||
|
||||
The `Query()` object referenced above should be made up of one or more of the query types listed below.
|
||||
So for example, a SetBit() query would look like this:
|
||||
```sh
|
||||
$ curl -X POST "http://127.0.0.1:10101/query?index=sample-idx" -d 'SetBit(project=10, frame="collaboration", user=1)'
|
||||
```
|
||||
|
||||
Query results have the format `{"results":[]}`, where `results` is a list of results for each `Query()`. This
|
||||
means that you can provide multiple `Query()` objects with each HTTP request and `results` will contain
|
||||
the results of all of the queries.
|
||||
|
||||
```sh
|
||||
$ curl -X POST "http://127.0.0.1:10101/query?index=sample-idx" -d 'Query() Query() Query()'
|
||||
```
|
||||
|
||||
---
|
||||
#### SetBit()
|
||||
```
|
||||
SetBit(project=10, frame="collaboration", user=1)
|
||||
```
|
||||
A return value of `{"results":[true]}` indicates that the bit was toggled from 0 to 1.
|
||||
A return value of `{"results":[false]}` indicates that the bit was already set to 1 and therefore nothing changed.
|
||||
|
||||
SetBit accepts an optional `timestamp` field:
|
||||
```
|
||||
SetBit(project=10, frame="collaboration", user=2, timestamp="2016-12-11T10:09:07")
|
||||
```
|
||||
|
||||
---
|
||||
#### ClearBit()
|
||||
```
|
||||
ClearBit(project=10, frame="collaboration", user=1)
|
||||
```
|
||||
A return value of `{"results":[true]}` indicates that the bit was toggled from 1 to 0.
|
||||
A return value of `{"results":[false]}` indicates that the bit was already set to 0 and therefore nothing changed.
|
||||
|
||||
---
|
||||
#### SetRowAttrs()
|
||||
```
|
||||
SetRowAttrs(project=10, frame="collaboration", stars=123, url="http://projects.pilosa.com/10", active=true)
|
||||
```
|
||||
Returns `{"results":[null]}`
|
||||
|
||||
---
|
||||
#### SetColumnAttrs()
|
||||
---
|
||||
```
|
||||
SetColumnAttrs(user=10, friends=123, username="mrpi", active=true)
|
||||
```
|
||||
|
||||
Returns `{"results":[null]}`
|
||||
|
||||
---
|
||||
#### Bitmap()
|
||||
```
|
||||
Bitmap(project=10, frame="collaboration")
|
||||
```
|
||||
Returns `{"results":[{"attrs":{"stars":123, "url":"http://projects.pilosa.com/10", "active":true},"bits":[1,2]}]}` where `attrs` are the
|
||||
attributes set using `SetRowAttrs()` and `bits` are the bits set using `SetBit()`.
|
||||
|
||||
In order to return column attributes attached to the columns of a bitmap, add `&columnAttrs=true` to the query string. Sample response:
|
||||
```
|
||||
{"results":[{"attrs":{},"bits":[10]}],"columnAttrs":[{"user":10,"attrs":{"friends":123, "username":"mrpi", "active":true}}]}
|
||||
```
|
||||
|
||||
---
|
||||
#### Union()
|
||||
```
|
||||
Union(Bitmap(project=10, frame="collaboration"), Bitmap(project=20, frame="collaboration")))
|
||||
```
|
||||
Returns a result set similar to that of a `Bitmap()` query, only the `attrs` dictionary will be empty: `{"results":[{"attrs":{},"bits":[1,2]}]}`.
|
||||
Note that a `Union()` query can be nested within other queries anywhere that you would otherwise provide a `Bitmap()`.
|
||||
|
||||
---
|
||||
#### Intersect()
|
||||
```
|
||||
Intersect(Bitmap(project=10, frame="collaboration"), Bitmap(project=20, frame="collaboration")))
|
||||
```
|
||||
Returns a result set similar to that of a `Bitmap()` query, only the `attrs` dictionary will be empty: `{"results":[{"attrs":{},"bits":[1]}]}`.
|
||||
Note that an `Intersect()` query can be nested within other queries anywhere that you would otherwise provide a `Bitmap()`.
|
||||
|
||||
---
|
||||
#### Difference()
|
||||
```
|
||||
Difference(Bitmap(project=10, frame="collaboration"), Bitmap(project=20, frame="collaboration")))
|
||||
```
|
||||
`Difference()` represents all of the bits that are set in the first `Bitmap()` but are not set in the second `Bitmap()`. It returns a result set similar to that of a `Bitmap()` query, only the `attrs` dictionary will be empty: `{"results":[{"attrs":{},"bits":[2]}]}`.
|
||||
Note that a `Difference()` query can be nested within other queries anywhere that you would otherwise provide a `Bitmap()`.
|
||||
|
||||
---
|
||||
#### Count()
|
||||
```
|
||||
Count(Bitmap(project=10, frame="collaboration"))
|
||||
```
|
||||
Returns the count of the number of bits set in `Bitmap()`: `{"results":[28]}`
|
||||
|
||||
---
|
||||
#### Range()
|
||||
```
|
||||
Range(project=10, frame="collaboration", start="1970-01-01T00:00", end="2000-01-02T03:04")
|
||||
```
|
||||
|
||||
---
|
||||
#### TopN()
|
||||
```
|
||||
TopN(frame="geo")
|
||||
```
|
||||
Returns all Bitmaps in the cache from frame `geo` sorted by the count of bits.
|
||||
|
||||
```
|
||||
TopN(frame="geo", n=20)
|
||||
```
|
||||
Returns the top 20 Bitmaps from frame `geo`.
|
||||
|
||||
```
|
||||
TopN(Bitmap(project=10, frame="collaboration"), frame="geo", n=20)
|
||||
```
|
||||
Returns the top 20 Bitmaps from `geo` sorted by the count of bits in the intersection with `Bitmap(project=10)`.
|
||||
|
||||
```
|
||||
TopN(Bitmap(project=10, frame="collaboration"), frame="geo", n=20, field="category", [81,82])
|
||||
```
|
||||
Returns the top 20 Bitmaps from `geo`in attribute `category` with values `81 or
|
||||
82` sorted by the count of bits in the intersection with `Bitmap(project=10)`.
|
||||
|
||||
## Development
|
||||
|
||||
### Updating dependencies
|
||||
|
||||
To update dependencies, you'll need to install [Glide][].
|
||||
|
||||
Then add the new dependencies in your project:
|
||||
|
||||
```sh
|
||||
$ glide get github.com/foo/bar
|
||||
```
|
||||
|
||||
### Protobuf
|
||||
|
||||
If you update protobuf (pilosa/internal/internal.proto), then you need to run `go generate`
|
||||
```sh
|
||||
$ go generate
|
||||
```
|
||||
|
||||
### Version
|
||||
|
||||
In order to set the version number, compile Pilosa with the following argument:
|
||||
```sh
|
||||
$ go install --ldflags="-X main.Version=1.0.0"
|
||||
```
|
||||
|
||||
[Glide]: http://glide.sh/
|
||||
Pilosa is an open source project. Please see our [Contributing Guide](https://www.pilosa.com/docs/contributing/) for information about how to get involved.
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue