Commit graph

568 commits

Author SHA1 Message Date
Ben Johnson
a8a2f8913c
add protobuf/testdata, fix Makefile
This commit re-adds the `testdata` folder that godeps strips and
also updates the `Makefile` to add a `vendor` target that will
handle this in the future.
2016-04-26 08:48:04 -06:00
Todd Gruben
0c8f9b3d87 Merge branch 'master' of github.com:umbel/pilosa 2016-04-19 09:24:07 -05:00
tgruben
ae7d26f8b4 Merge pull request #73 from benbjohnson/anti-entropy-monitor
Anti-entropy monitor
2016-04-19 09:23:56 -05:00
Todd Gruben
cc048f5f3f made Bitmap SetBit method public 2016-04-19 09:22:42 -05:00
Ben Johnson
e925c10bb8 Merge pull request #72 from travisturner/docs
fixed typos. changed -config flag to represent prefered Go format
2016-04-18 16:45:18 -06:00
Ben Johnson
f29ad7dfa8
add anti-entropy monitor 2016-04-18 16:32:27 -06:00
travisturner
5ab025638b fixed typos. changed -config flag to represent prefered Go format 2016-04-15 12:50:47 -05:00
tgruben
847f5a5eb0 Merge pull request #71 from benbjohnson/anti-entropy
Active Anti-Entropy (AAE)
2016-04-13 14:49:45 -05:00
tgruben
70f299d3f9 Merge pull request #70 from travisturner/docs
update the README file with config and query information
2016-04-13 10:18:05 -05:00
Ben Johnson
37d90eac7a
active anti-entropy
This commit adds active anti-entropy via a merkle tree structure.
2016-04-12 19:48:32 -06:00
travisturner
c591ee0f9a update the README file with config and query information 2016-04-11 22:28:50 -05:00
tgruben
37f18bb3f5 Merge pull request #68 from benbjohnson/bench
Add 'pilosactl bench' command
2016-04-01 12:17:18 -05:00
Ben Johnson
dfda274c40 add 'pilosactl bench' command
This commit adds a simple benchmarking utility to the `pilosactl`
binary. It currently only supports individual `SetBit()` commands
but it's a good start towards making a generic benchmarking
framework at the integration level.

The subcommands and usage/help messages were also cleaned up to
output correctly.
2016-03-31 15:49:57 -06:00
tgruben
e52b037226 Merge pull request #67 from benbjohnson/op-n
Snapshot after op threshold
2016-03-31 09:28:03 -05:00
Ben Johnson
7f1e7e91c7 snapshot after op threshold
This commit changes `SetBit()` and `ClearBit()` to issue a
snapshot after a certain number of calls are made on a given
fragment.

Fixes #66
2016-03-30 15:28:42 -06:00
tgruben
0cb961a5b5 Merge pull request #65 from benbjohnson/schema
Add schema endpoint
2016-03-28 16:15:16 -05:00
Ben Johnson
06e048def6 add schema endpoint
This commit adds a new HTTP handler to return a list of all databases
and frames in the index:

	GET /schema

This returns JSON in the following format:

	{
	  "dbs":[
	    {
	      "name":"d0",
	      "frames":[
	        {"name":"f0"},
	        {"name":"f1"}
	      ]
	    }
	  ]
	}

Fixes #61
2016-03-22 11:25:52 -06:00
tgruben
81a885e557 Merge pull request #64 from benbjohnson/backup-restore-cache
Add cache to fragment backup/restore
2016-03-18 09:47:38 -05:00
Ben Johnson
137247311b add cache to fragment backup/restore
This commit changes the backup format of a fragment from a
simple file stream to a tar archive which combines the data file
and the cache file.

Fixes #63
2016-03-17 15:48:48 -06:00
tgruben
1af478d78f Merge pull request #62 from benbjohnson/frame-restore
Add frame restore for cluster transfer
2016-03-16 09:25:01 -05:00
Ben Johnson
659da33ea2 add frame restore for cluster transfer
This commit adds an endpoint to restore an entire frame from
another cluster. Multiple hosts can use this endpoint to copy
and rebalance a cluster to a new cluster.
2016-03-15 14:42:42 -06:00
tgruben
8b86901d90 Merge pull request #60 from benbjohnson/multi-call
Add bulk call support to the query endpoint
2016-03-09 10:53:07 -06:00
Ben Johnson
ee521e2bbf add bulk call support to the query endpoint
This commit changes the `pql.Query` so that it can accept one or more
top-level calls instead of only one.

The query request format change because a query with a single call is
still valid. However, the result format now returns a `results` field
that has one result for each top-level call. The `profiles` field is
still the same, however, it combines all profiles from all bitmap
responses into one return so that there's not duplicate attributes.

Fixes #59
2016-03-08 14:34:37 -07:00
tgruben
e7b47efd8d Merge pull request #58 from benbjohnson/top-n-2
Refetch full counts for TopN()
2016-03-03 14:43:56 -06:00
Ben Johnson
9f85d82e3a refetch full counts for TopN()
This commits adds refetching to get the actual TopN() count
after the top bitmaps are determined.

Fixes #57
2016-03-03 13:03:20 -07:00
tgruben
84a51fbd66 Merge pull request #56 from benbjohnson/backup
Add backup/restore to pilosactl
2016-02-29 10:49:58 -06:00
Todd Gruben
e028cc25ed test for topn fill 2016-02-29 10:18:57 -06:00
Ben Johnson
cfa1a71190 add backup/restore to pilosactl
This commit adds two commands:

    pilosactl backup
    pilosactl restore

The `Client` implementations have also been added to support these
commands:

    func (c *Client) BackupTo(w io.Writer, db, frame string) error
    func (c *Client) RestoreFrom(r io.Reader, db, frame string) error

Backups occur on a per-frame basis and all slices from the cluster
are packed into a single tar file. The backup tool attempts to
read from owner nodes in a random order and will retry against
the next owner if one fails.

During restore, the slices are restored to all owner nodes. This
allows users to backup a frame from one cluster and restore it
to a different one -- even if the topology of the new cluster is
different.
2016-02-29 08:38:24 -07:00
tgruben
a12ec57eec Merge pull request #54 from umbel/todd
fixed issue with SetBit/ClearBit not correctly reporting bitchange
2016-02-26 12:13:41 -06:00
Todd Gruben
ab5f436e44 merge 2016-02-26 12:13:03 -06:00
tgruben
794dccc854 Merge pull request #55 from benbjohnson/backup-restore
Add backup/restore endpoints
2016-02-25 18:37:40 -06:00
Ben Johnson
acdcbacad7 add backup/restore endpoints
This commit adds io.ReaderFrom and io.WriterTo implementations to
the Fragment and also adds backup & restore endpoints to the Handler.
2016-02-25 17:29:45 -07:00
Todd Gruben
daf828d2a8 generated using go generate 2016-02-25 17:15:40 -06:00
Todd Gruben
c60773d33a issue 41 2016-02-25 15:43:52 -06:00
Todd Gruben
1aca83a546 align more with go idioms 2016-02-24 15:02:24 -06:00
Todd Gruben
a72edfdae8 merged 2016-02-24 14:19:44 -06:00
Todd Gruben
ed1890b3a2 adjust SetBit/ClearBit to releably report bit change 2016-02-24 14:00:41 -06:00
Ben Johnson
7ef7eee569 Merge pull request #52 from benbjohnson/range
Add Range() support
2016-02-23 14:59:03 -07:00
Ben Johnson
42ba865b6d add Range() support
This commit adds support for setting bits in time-based frames.
2016-02-23 14:53:03 -07:00
tgruben
faf6406791 Merge pull request #50 from umbel/todd
added support for returning SetBit effect
2016-02-22 11:36:22 -06:00
Todd Gruben
7d6a8846b5 cleaned up tests for SetBit 2016-02-22 11:30:06 -06:00
Todd Gruben
a74d826b41 removed debugging code' 2016-02-17 13:23:30 -06:00
Ben Johnson
e1eca36a9d fix int64 attr 2016-02-17 10:59:17 -07:00
Todd Gruben
6d3d10da2b changed protobuf to uint; doesn't work 2016-02-17 11:06:57 -06:00
Todd Gruben
7c8a27338a added support for returning SetBit effect 2016-02-16 17:38:56 -06:00
tgruben
00413a9e9a Merge pull request #49 from benbjohnson/attr
Refactor attribute stores
2016-02-16 16:27:22 -06:00
Ben Johnson
b873b4c084 refactor attribute stores
This commit refactors the profile and bitmap attribute stores so
that they share the same code. There is a new `AttrStore` which
associates key/value pairs with a `uint64` identifier.

The previous JSON encoding has been fixed to use protobufs which
fixes encoding issues for int64.
2016-02-16 13:41:37 -07:00
tgruben
2b5831cbf2 Merge pull request #47 from umbel/proto-testdata
Re-add proto/testdata
2016-02-15 10:26:35 -06:00
Ben Johnson
985613e6e1 add proto/testdata 2016-02-15 09:25:14 -07:00
tgruben
22c3ac5aa4 Merge pull request #46 from benbjohnson/cache-persistence
Add cache persistence
2016-02-15 09:29:57 -06:00