diff --git a/README.md b/README.md index acab512eb..3fc7e4c56 100644 --- a/README.md +++ b/README.md @@ -1,326 +1,61 @@ -# pilosa - -Pilosa is a bitmap index. + + + [![Build Status](https://travis-ci.com/pilosa/pilosa.svg?token=Peb4jvQ3kLbjUEhpU5aR&branch=master)](https://travis-ci.com/pilosa/pilosa) +## An open source, distributed bitmap index. +- [Docs](#docs) +- [Getting Started](#getting-started) +- [Data Model](#data-model) +- [Client Libraries](#client-drivers) +- [Get Support](#get-support) +- [Contributing](#contributing) + + +## Docs + +See our [Documentation](https://www.pilosa.com/docs/) for information about installing and working with Pilosa. + + ## Getting Started -Pilosa requires Go 1.7 or greater. +[Getting Started](https://www.pilosa.com/docs/getting-started/) -You can download the source by running `go get`: -```sh -$ go get github.com/pilosa/pilosa -``` +1. [Install Pilosa](https://www.pilosa.com/docs/installation/). -Now you can install the `pilosa` binary: +2. [Start Pilosa](https://www.pilosa.com/docs/getting-started/#starting-pilosa) with the default configuration: -```sh -$ go install github.com/pilosa/pilosa/cmd/... -``` + ```shell + pilosa server + ``` + + and verify that it's running: + + ```shell + curl localhost:10101/nodes + ``` -Now run a single pilosa node with the default configuration: +3. Follow along with the [Sample Project](https://www.pilosa.com/docs/getting-started/#sample-project) to get a better understanding of Pilosa's capabilities. -```sh -pilosa server -``` -## Configuration +## Data Model -Running just `pilosa` will show a list of available subcommands. `pilosa help -` will show usage information and the available flags for the command. +Check out how the Pilosa [Data Model](https://www.pilosa.com/docs/data-model/) works. -Any flag can be specified at the command line, in an environment variables, -and/or in a toml config file. The environment variable for any flag is that -flag, upper cased, prefixed with `PILOSA_`, and with any dashes converted to -underscores. For example, if you would specify `--cluster.poll-interval=30s` at -the command line, you would set `PILOSA_CLUSTER.POLL_INTERVAL=30s` in the -environment. For the configuration file, a dot in a flag denotes nesting with in -the config file. See the example config file below for examples of this. -You can specify a configuration by setting the `--config` flag when running -`pilosa`. +## Client Libraries +There are supported libraries for the following languages: +- [Go](https://www.pilosa.com/docs/client-libraries/#go) +- [Java](https://www.pilosa.com/docs/client-libraries/#java) +- [Python](https://www.pilosa.com/docs/client-libraries/#python) -```sh -pilosa server --config custom-config-file.cfg -``` +## Get Support -The config file uses the [TOML](https://github.com/toml-lang/toml) configuration file format, -and should look like: +There are [several channels](https://www.pilosa.com/community/#support) availble for you to reach out to us for support. -``` -data-dir = "/tmp/pil0" -bind = "127.0.0.1:10101" +## Contributing -[cluster] - poll-interval = "2m0s" - replicas = 2 - hosts = [ - "127.0.0.1:10101", - "127.0.0.1:10102", - ] - -[anti-entropy] - interval = "10m0s" - -[profile] - cpu = "/home/mycpuprofile" - cpu-time = "30s" -``` - -You can generate a template config file with default values with: - -```sh -pilosa config -``` - -The first two configuration options will be unique to each node in the cluster: - -`data-dir`: directory in which data is stored to disk - -`bind`: IP and port that pilosa will listen on - -The remaining configuration options should be the same on every node in the cluster. - -`[cluster] replicas`: the number of replicas within the cluster - -`[cluster] hosts`: specifies each node within the cluster - -`[cluster] poll-interval`: TODO - -`[anti-entropy] interval`: TODO - -There are also some profiling options for debugging and performance tuning - these don't need to be the same across the cluster and are mostly useful for doing Pilosa development. - -`[profile] cpu`: Path at which to store cpu profiling data which will be taken when pilosa starts. - -`[profile] cpu-time`: Amount of time for which to collect cpu profiling data at startup. - -## Docker - -You can create a Pilosa container using `make docker` or equivalently: -``` -docker build -t pilosa:latest . -``` - -You can run a temporary container using: -``` -docker run -it --rm --name pilosa -p 10101:10101 pilosa:latest -``` - -When you click `Ctrl+C` to stop the container, the container and the data in the container will be erased. You can leave out `--rm` flag to keep the data in the container. See [Docker documentation](https://docs.docker.com) for other options. - -## Usage - -You can interact with Pilosa via HTTP requests to the host:port on which you have Pilosa running. -The following examples illustrate how to do this using `curl` with a Pilosa cluster running on -127.0.0.1 port 10101. - -Return the version of Pilosa: -```sh -$ curl "http://127.0.0.1:10101/version" -``` - -Return a list of all indexes and frames in the index: -```sh -$ curl "http://127.0.0.1:10101/schema" -``` - -### Index and Frame Schema - -Before running a query, the corresponding index and frame must be created. Note that index and frame names can contain only lower case letters, numbers, dash (`-`), underscore (`_`) and dot (`.`). - -You can create the index `sample-idx` using: - -```sh -$ curl -XPOST "http://127.0.0.1:10101/index" \ - -d '{"index": "sample-idx"}' -``` - -Optionally, you can specify the column label on index creation: - -```sh -$ curl -XPOST "http://127.0.0.1:10101/index" \ - -d '{"index": "sample-idx", "options": {"columnLabel": "user"}}' -``` - -The frame `collaboration` may be created using the following call: - -```sh -$ curl -XPOST "http://127.0.0.1:10101/frame" \ - -d '{"index": "sample-idx", "frame": "collaboration"}' -``` - -It is possible to specify the frame row label on frame creation: - -```sh -$ curl -XPOST "http://127.0.0.1:10101/frame" \ - -d '{"index": "sample-idx", "frame": "collaboration", "options": {"rowLabel": "project"}}' -``` - -### Queries - -Queries to Pilosa require sending a POST request where the query itself is sent as POST data. -You specify the index on which to perform the query with a URL argument `index=index-name`. - -In this section, we assume both the index `sample-idx` with column label `user` and the frame `collaboration` with row label `project` was created. - -A query sent to index `sample-idx` will have the following format: - -```sh -$ curl -X POST "http://127.0.0.1:10101/query?index=sample-idx" -d 'Query()' -``` - -The `Query()` object referenced above should be made up of one or more of the query types listed below. -So for example, a SetBit() query would look like this: -```sh -$ curl -X POST "http://127.0.0.1:10101/query?index=sample-idx" -d 'SetBit(project=10, frame="collaboration", user=1)' -``` - -Query results have the format `{"results":[]}`, where `results` is a list of results for each `Query()`. This -means that you can provide multiple `Query()` objects with each HTTP request and `results` will contain -the results of all of the queries. - -```sh -$ curl -X POST "http://127.0.0.1:10101/query?index=sample-idx" -d 'Query() Query() Query()' -``` - ---- -#### SetBit() -``` -SetBit(project=10, frame="collaboration", user=1) -``` -A return value of `{"results":[true]}` indicates that the bit was toggled from 0 to 1. -A return value of `{"results":[false]}` indicates that the bit was already set to 1 and therefore nothing changed. - -SetBit accepts an optional `timestamp` field: -``` -SetBit(project=10, frame="collaboration", user=2, timestamp="2016-12-11T10:09:07") -``` - ---- -#### ClearBit() -``` -ClearBit(project=10, frame="collaboration", user=1) -``` -A return value of `{"results":[true]}` indicates that the bit was toggled from 1 to 0. -A return value of `{"results":[false]}` indicates that the bit was already set to 0 and therefore nothing changed. - ---- -#### SetRowAttrs() -``` -SetRowAttrs(project=10, frame="collaboration", stars=123, url="http://projects.pilosa.com/10", active=true) -``` -Returns `{"results":[null]}` - ---- -#### SetColumnAttrs() ---- -``` -SetColumnAttrs(user=10, friends=123, username="mrpi", active=true) -``` - -Returns `{"results":[null]}` - ---- -#### Bitmap() -``` -Bitmap(project=10, frame="collaboration") -``` -Returns `{"results":[{"attrs":{"stars":123, "url":"http://projects.pilosa.com/10", "active":true},"bits":[1,2]}]}` where `attrs` are the -attributes set using `SetRowAttrs()` and `bits` are the bits set using `SetBit()`. - -In order to return column attributes attached to the columns of a bitmap, add `&columnAttrs=true` to the query string. Sample response: -``` -{"results":[{"attrs":{},"bits":[10]}],"columnAttrs":[{"user":10,"attrs":{"friends":123, "username":"mrpi", "active":true}}]} -``` - ---- -#### Union() -``` -Union(Bitmap(project=10, frame="collaboration"), Bitmap(project=20, frame="collaboration"))) -``` -Returns a result set similar to that of a `Bitmap()` query, only the `attrs` dictionary will be empty: `{"results":[{"attrs":{},"bits":[1,2]}]}`. -Note that a `Union()` query can be nested within other queries anywhere that you would otherwise provide a `Bitmap()`. - ---- -#### Intersect() -``` -Intersect(Bitmap(project=10, frame="collaboration"), Bitmap(project=20, frame="collaboration"))) -``` -Returns a result set similar to that of a `Bitmap()` query, only the `attrs` dictionary will be empty: `{"results":[{"attrs":{},"bits":[1]}]}`. -Note that an `Intersect()` query can be nested within other queries anywhere that you would otherwise provide a `Bitmap()`. - ---- -#### Difference() -``` -Difference(Bitmap(project=10, frame="collaboration"), Bitmap(project=20, frame="collaboration"))) -``` -`Difference()` represents all of the bits that are set in the first `Bitmap()` but are not set in the second `Bitmap()`. It returns a result set similar to that of a `Bitmap()` query, only the `attrs` dictionary will be empty: `{"results":[{"attrs":{},"bits":[2]}]}`. -Note that a `Difference()` query can be nested within other queries anywhere that you would otherwise provide a `Bitmap()`. - ---- -#### Count() -``` -Count(Bitmap(project=10, frame="collaboration")) -``` -Returns the count of the number of bits set in `Bitmap()`: `{"results":[28]}` - ---- -#### Range() -``` -Range(project=10, frame="collaboration", start="1970-01-01T00:00", end="2000-01-02T03:04") -``` - ---- -#### TopN() -``` -TopN(frame="geo") -``` -Returns all Bitmaps in the cache from frame `geo` sorted by the count of bits. - -``` -TopN(frame="geo", n=20) -``` -Returns the top 20 Bitmaps from frame `geo`. - -``` -TopN(Bitmap(project=10, frame="collaboration"), frame="geo", n=20) -``` -Returns the top 20 Bitmaps from `geo` sorted by the count of bits in the intersection with `Bitmap(project=10)`. - -``` -TopN(Bitmap(project=10, frame="collaboration"), frame="geo", n=20, field="category", [81,82]) -``` -Returns the top 20 Bitmaps from `geo`in attribute `category` with values `81 or -82` sorted by the count of bits in the intersection with `Bitmap(project=10)`. - -## Development - -### Updating dependencies - -To update dependencies, you'll need to install [Glide][]. - -Then add the new dependencies in your project: - -```sh -$ glide get github.com/foo/bar -``` - -### Protobuf - -If you update protobuf (pilosa/internal/internal.proto), then you need to run `go generate` -```sh -$ go generate -``` - -### Version - -In order to set the version number, compile Pilosa with the following argument: -```sh -$ go install --ldflags="-X main.Version=1.0.0" -``` - -[Glide]: http://glide.sh/ +Pilosa is an open source project. Please see our [Contributing Guide](https://www.pilosa.com/docs/contributing/) for information about how to get involved.