No description
Find a file
Seebs 3b696da34a plugins and precomputed data
So in some cases, when we do a query, the results of one
part of the query are innately shared-across-nodes; for
instance, a hypothetical Distinct query. More generally,
we allow cross-index queries; calls can have "index=foo"
in them.

This patch lets us handle that without duplicating that
query all over. Before we actually start doing the
separate calls, we run the query once from the coordinating
node, then patch the results in, and send relevant subsets
over to each client, etcetera. Also provides slightly
friendlier (and I hope faster) support for converting
bitmaps to/from sets of rows.

We also add an extension interface, and some fancy stuff
to let us define new calls, which use this. They're sort
of tied together because the first extension I wanted to
implement needed precomputed calls. The extension API
lets us create extensions using `pkg/plugin` (with all its
associated limitations, unfortunately), then query them
at load time for functionality.

This also implies some revamping of the argument
validation for PQL, like verifying that functions exist
and knowing things about their argument types.

So basically this is an overly intrusive patch, and would
be better as separate patches, but they're hard to detangle.

add trivial execution-time profiling

What if you could ?profile=true on a query and get some
numbers back? That'd be really cool.

We already have tracing/spans, but right now, those only generate
any data if you have something set up for them to trace to. Add a
fancy wrapper that lets us generate our own tracing data, and dump
it into the request response, if ?profile=true.

add a sample extension, add missing features to extension interface

Implement a naive probabilistic filter extension as an example of
what an extension looks like. In the process, discover multiple
omissions in the bitmap API. Well, I did *say* it was experimental.
2019-11-12 12:14:29 -06:00
.circleci version-bump golangci-lint 2019-10-11 14:50:37 -05:00
.github Add changelog steps to PR template 2019-04-12 13:47:40 -05:00
api/client use TLS settings when setting up GRPC server or client 2019-10-30 17:38:48 -05:00
boltdb Translation store refactor 2019-10-09 08:59:41 -06:00
cmd support for decimal fields 2019-10-29 16:36:14 -05:00
ctl fix golangci-lint complaints 2019-11-10 17:58:46 -06:00
docs Add support for TLS client certificate verification and custom CA 2019-09-24 11:09:58 -05:00
encoding/proto plugins and precomputed data 2019-11-12 12:14:29 -06:00
enterprise drop enterprise/b 2019-05-30 16:36:20 -05:00
ext plugins and precomputed data 2019-11-12 12:14:29 -06:00
gcnotify v2.0.0 2019-10-08 14:56:17 -06:00
gopsutil v2.0.0 2019-10-08 14:56:17 -06:00
gossip v2.0.0 2019-10-08 14:56:17 -06:00
http reset fragment.rowCache after importValue 2019-11-11 16:54:51 -06:00
internal plugins and precomputed data 2019-11-12 12:14:29 -06:00
logger so much lint 2019-04-16 12:07:18 -05:00
lru so much lint 2019-04-16 12:07:18 -05:00
mock Translation store refactor 2019-10-09 08:59:41 -06:00
pql plugins and precomputed data 2019-11-12 12:14:29 -06:00
prometheus v2.0.0 2019-10-08 14:56:17 -06:00
proto Change proto package name to "pilosa" to not conflict with molecula. Upgrade protoc to 3.10.1 2019-11-11 12:24:38 -06:00
roaring plugins and precomputed data 2019-11-12 12:14:29 -06:00
server fix golangci-lint complaints 2019-11-10 17:58:46 -06:00
shardwidth Fix enterprise license header and add shardwidth files to license header check 2019-04-12 11:57:28 -05:00
stats v2.0.0 2019-10-08 14:56:17 -06:00
statsd v2.0.0 2019-10-08 14:56:17 -06:00
syswrap Add license headers to files missing them and CI check to verify they are present. Fixes #1633 2019-04-12 11:30:41 -05:00
test initial gRPC server implementation 2019-10-29 14:27:07 -05:00
testdata/sample_view Adds fragment test data, implements #1090 2018-05-01 17:28:10 +03:00
toml Add license headers to files missing them and CI check to verify they are present. Fixes #1633 2019-04-12 11:30:41 -05:00
tracing add trivial execution-time profiling 2019-10-29 15:23:37 -05:00
.gitignore Hide build directory in .gitignore 2017-05-19 14:17:57 -05:00
.golangci.yml fix golangci-lint complaints 2019-11-10 17:58:46 -06:00
api.go plugins and precomputed data 2019-11-12 12:14:29 -06:00
api_test.go support for decimal fields 2019-10-29 16:36:14 -05:00
apimethod_string.go send POSTed schema to all nodes in cluster 2019-04-29 19:31:23 -05:00
attr.go v2.0.0 2019-10-08 14:56:17 -06:00
attr_test.go v2.0.0 2019-10-08 14:56:17 -06:00
broadcast.go Fix linter issues: gochecknoinits 2018-07-18 11:58:27 -05:00
cache.go v2.0.0 2019-10-08 14:56:17 -06:00
cache_test.go v2.0.0 2019-10-08 14:56:17 -06:00
CHANGELOG.md Fix typo 2019-09-17 16:01:21 -05:00
client.go support for decimal fields 2019-10-29 16:36:14 -05:00
cluster.go Translation store refactor 2019-10-09 08:59:41 -06:00
cluster_internal_test.go Sources and Generations: tracking mmapped files 2019-11-12 12:14:29 -06:00
cmd.go Copy dependency into project and use pilosa's logger 2019-10-03 09:44:48 -05:00
CONTRIBUTING.md simplify contributing instructions by removing weird upstream thing 2019-05-01 17:37:52 -05:00
diagnostics.go support for decimal fields 2019-10-29 16:36:14 -05:00
diagnostics_internal_test.go v2.0.0 2019-10-08 14:56:17 -06:00
doc.go Apply Apache License 2.0 2017-04-28 14:22:03 -05:00
Dockerfile Release v1.4.0 2019-09-17 14:47:59 -05:00
Dockerfile-clustertests Add go module support 2019-01-29 14:53:03 -06:00
event.go fix some lint warnings raised in VS-Code 2019-04-17 18:10:05 -05:00
executor.go plugins and precomputed data 2019-11-12 12:14:29 -06:00
executor_internal_test.go Translation store refactor 2019-10-09 08:59:41 -06:00
executor_test.go Add an IncludesColumn() function to PQL 2019-11-11 08:17:28 -06:00
extension.go plugins and precomputed data 2019-11-12 12:14:29 -06:00
field.go handle truncated/damaged .available.shards 2019-11-12 12:14:29 -06:00
field_internal_test.go handle truncated/damaged .available.shards 2019-11-12 12:14:29 -06:00
field_test.go v2.0.0 2019-10-08 14:56:17 -06:00
fragment.go Sources and Generations: tracking mmapped files 2019-11-12 12:14:29 -06:00
fragment_internal_test.go Sources and Generations: tracking mmapped files 2019-11-12 12:14:29 -06:00
gc.go fix some lint warnings raised in VS-Code 2019-04-17 18:10:05 -05:00
generation.go Sources and Generations: tracking mmapped files 2019-11-12 12:14:29 -06:00
generation_debug.go Sources and Generations: tracking mmapped files 2019-11-12 12:14:29 -06:00
generation_nodebug.go Sources and Generations: tracking mmapped files 2019-11-12 12:14:29 -06:00
generation_test.go Sources and Generations: tracking mmapped files 2019-11-12 12:14:29 -06:00
go.mod plugins and precomputed data 2019-11-12 12:14:29 -06:00
go.sum plugins and precomputed data 2019-11-12 12:14:29 -06:00
handler.go plugins and precomputed data 2019-11-12 12:14:29 -06:00
holder.go Translation store refactor 2019-10-09 08:59:41 -06:00
holder_internal_test.go v2.0.0 2019-10-08 14:56:17 -06:00
holder_test.go Sources and Generations: tracking mmapped files 2019-11-12 12:14:29 -06:00
index.go use labeled targets for break statements 2019-10-11 14:44:25 -05:00
index_internal_test.go make sure more tests and benchmarks can have their temp dir set by flag 2019-01-21 16:23:27 -06:00
index_test.go v2.0.0 2019-10-08 14:56:17 -06:00
iterator.go v2.0.0 2019-10-08 14:56:17 -06:00
iterator_internal_test.go rename internal_tests. fix license header 2018-07-04 10:59:08 -05:00
LICENSE Apply Apache License 2.0 2017-04-28 14:22:03 -05:00
license.exceptions add proto/pilosa.pb.go to license.exceptions list 2019-11-10 11:33:29 -06:00
Makefile add decimal field support to Inspect 2019-10-30 11:13:48 -05:00
NOTES rename slice to shard 2018-06-28 14:07:07 -05:00
NOTICE Update license information 2019-10-03 12:20:52 -05:00
pilosa.go Add min/max constraints; fix tests 2019-05-19 16:05:22 -06:00
pilosa_internal_test.go change exists field to _exists 2018-12-17 15:44:50 -06:00
pilosa_test.go v2.0.0 2019-10-08 14:56:17 -06:00
README.md Remove TravisCI, add CircleCI shield 2018-11-13 12:34:44 -06:00
row.go plugins and precomputed data 2019-11-12 12:14:29 -06:00
row_test.go Add an IncludesColumn() function to PQL 2019-11-11 08:17:28 -06:00
server.go plugins and precomputed data 2019-11-12 12:14:29 -06:00
server_internal_test.go remove shard validation stuff 2019-04-22 17:36:40 -05:00
time.go add logic to restrict time range to available views 2019-02-01 15:30:10 -06:00
time_internal_test.go add logic to restrict time range to available views 2019-02-01 15:30:10 -06:00
translate.go use labeled targets for break statements 2019-10-11 14:44:25 -05:00
translator_test.go Translation store refactor 2019-10-09 08:59:41 -06:00
uri.go Translation store refactor 2019-10-09 08:59:41 -06:00
uri_internal_test.go Require a valid port that isn't greater than 65,535 2018-08-16 21:05:41 -04:00
utils_internal_test.go Sources and Generations: tracking mmapped files 2019-11-12 12:14:29 -06:00
version.go Fix linter issues: gochecknoinits 2018-07-18 11:58:27 -05:00
view.go Sources and Generations: tracking mmapped files 2019-11-12 12:14:29 -06:00
view_internal_test.go view.go: deal with races in fragment creation 2019-02-21 16:43:19 -06:00

CircleCI GoDoc Go Report Card license CLA Assistant GitHub release

An open source, distributed bitmap index.

Want to contribute? One of the easiest ways is to tell us how you're using (or want to use) Pilosa. We learn from every discussion!

Docs

See our Documentation for information about installing and working with Pilosa.

Getting Started

  1. Install Pilosa.

  2. Start Pilosa with the default configuration:

    pilosa server
    

    and verify that it's running:

    curl localhost:10101/nodes
    
  3. Follow along with the Sample Project to get a better understanding of Pilosa's capabilities.

Data Model

Check out how the Pilosa Data Model works.

Query Language

You can interact with Pilosa directly in the console using the Pilosa Query Language (PQL).

Client Libraries

There are supported libraries for the following languages:

Licenses

The core Pilosa code base and all default builds (referred to as Pilosa Community Edition) are licensed completely under the Apache License, Version 2.0. If you build Pilosa with the enterprise build tag (Pilosa Enterprise Edition), then that build will include features licensed under the GNU Affero General Public License (AGPL). Enterprise code is located entirely in the github.com/pilosa/pilosa/enterprise directory. See github.com/pilosa/pilosa/NOTICE and github.com/pilosa/pilosa/LICENSE for more information about Pilosa licenses.

Get Support

There are several channels available for you to reach out to us for support.

Contributing

Pilosa is an open source project. Please see our Contributing Guide for information about how to get involved.