Commit graph

1152 commits

Author SHA1 Message Date
Linh Vo
1be7a42d14 custom Unmarshal JSON 2017-04-04 14:50:43 -05:00
Linh Vo
abd696ca56 rm govalidation phase 2017-04-03 10:50:19 -05:00
Linh Vo
c034293967 validate db and frame 2017-04-02 22:26:36 -05:00
Ben Johnson
261e327014 Merge pull request #412 from benbjohnson/max-inverse-slices
Separate max slices for standard and inverse views.
2017-03-29 14:53:41 -06:00
Ben Johnson
912fcb3464 Separate max slices for standard and inverse views.
Previously only the standard max slice was available. This commit
changes it so that the inverse max can be retrieved separately
through the `DB` and `Frame` types as well as through the
`HTTP` API and `Client`.
2017-03-28 13:01:04 -06:00
Linh Vo
07ce417f55 Merge pull request #410 from linhvo/378-set-profile-attr
SetProfilesAttrs with option column
2017-03-27 10:29:07 -05:00
Ben Johnson
ed09d0474f Merge pull request #400 from benbjohnson/views
Separate physical data layout with views.
2017-03-24 16:33:29 -06:00
Ben Johnson
071f1f2652 Fix rebase. 2017-03-24 14:01:01 -06:00
Ben Johnson
3a71003240 Re-sort inverted bits on import. 2017-03-24 13:57:02 -06:00
Ben Johnson
eff08af112 Separate physical data layout with views.
Previously, multiple frames with different prefixes were used to separate
different data layouts. This included separating standard row/column
layouts from inverted column/row layouts as well as storing aggregate
information for timestamp data.

Unfortunately, this caused frame meta data to be copied between multiple
frames and it made it difficult to keep these frames in sync.

This commit separates these different physical layouts into `Views`.
A `Frame` now has one or more views which represent each layout.
Fragments have been moved from under the `Frame` to be contained
within the `View`.

There are two primary views:

- `standard`
- `inverse`

If a frame has a time quantum, then views are generated for these
each of the standard/inverse views. For example a time quantum
of `YMDH` for the date `2000-01-02T00:00:00Z` would create the
following views:

- `standard_2000`
- `inverse_2000`
- `standard_200001`
- `inverse_200001`
- `standard_20000102`
- `inverse_20000102`

From the user's perspective, nothing should change in PQL. Different
PQL statements will handle the appropriate view automatically. For
example, `Bitmap()` and `Profile()` will fetch using the `standard`
view or the `inverse` view, respectively. The `Range()` statement
will lookup the appropriate time-based views automatically.
2017-03-24 13:57:02 -06:00
Linh Vo
292302f60b fix test 2017-03-24 13:45:00 -05:00
Linh Vo
22bac56dc5 test setProfileAttrs with option column 2017-03-24 12:26:49 -05:00
Linh Vo
7979b4d69b set profile attr for columnLabel 2017-03-24 12:18:59 -05:00
Linh Vo
74b5f99e15 Merge pull request #409 from linhvo/fix-instructions
fix README
2017-03-24 10:17:09 -05:00
Linh Vo
318a8a9f19 fix README 2017-03-24 09:56:14 -05:00
Linh Vo
b0c2c08b2a Merge pull request #407 from linhvo/311-topn-return-id
use id instead of key in topN results
2017-03-23 14:01:46 -05:00
Linh Vo
ac8018c4f3 change Pair struct Key to ID 2017-03-23 12:38:09 -05:00
Linh Vo
67a691dfe0 use id instead of key in topN results 2017-03-23 12:02:57 -05:00
Linh Vo
d93abb8d07 Merge pull request #369 from linhvo/tanimoto
add tanimoto option to pilosa
2017-03-23 10:45:44 -05:00
Matthew Jaffee
6e082df9e5 Merge pull request #397 from jaffee/config-readme-tweaks
some more readme fixes for the new config style.
2017-03-20 11:23:05 -05:00
Matthew Jaffee
49efc558e6 fix single dash config flag in readme 2017-03-20 11:16:30 -05:00
Matt Jaffee
9b88e6e754 some more readme fixes for the new config style. 2017-03-20 11:09:19 -05:00
Matthew Jaffee
a3462a2290 Merge pull request #396 from jaffee/fix-readme-config
fix readme and config command for new config style
2017-03-20 08:34:37 -05:00
Matt Jaffee
77c25eeb06 fix readme and config command for new config style
still need to update readme to include info about environment variables, but at
least it isn't wrong anymore.
2017-03-17 20:42:41 -05:00
Matthew Jaffee
3bf3bf0aa4 Merge pull request #394 from jaffee/365-remove-pilosa-bench
remove pilosa-bench
2017-03-17 17:03:32 -05:00
Matt Jaffee
4c693ba7e7 remove pilosa-bench 2017-03-17 16:59:08 -05:00
Matt Jaffee
60805d7f3e change port 15000 to 10101 everywhere 2017-03-17 16:43:04 -05:00
Matt Jaffee
176861d702 add embedded struct to contain io for commands
per @tgruben's suggestion
2017-03-17 16:43:04 -05:00
Matt Jaffee
5f267c0a3a support the various ways of specifying bind addr 2017-03-17 16:43:04 -05:00
Matt Jaffee
74b79dd928 add tests for subcommands 2017-03-17 16:43:04 -05:00
Matt Jaffee
9aac0d143f add tests for backup cmd 2017-03-17 16:43:04 -05:00
Matt Jaffee
a1d29537db small tweaks - make dry-run hidden 2017-03-17 16:43:04 -05:00
Matt Jaffee
06b90782af remove ConfigPath from server (no longer used) 2017-03-17 16:43:04 -05:00
Matt Jaffee
e144259a19 make config flag global, and add global dry run flag
config flag can be used by all subcommands. Dry run flag stops subcommand
execution after parsing of config - nice for testing.
2017-03-17 16:43:04 -05:00
Matt Jaffee
dcafca3141 bind to localhost:0 in tests to avoid binding to the same port 2017-03-17 16:43:04 -05:00
Matt Jaffee
2aef65989e convert all commands to use RunE and global Command var 2017-03-17 16:43:04 -05:00
Matt Jaffee
601e8478da put all cmd test helpers in root_test and return errors from them 2017-03-17 16:43:04 -05:00
Matt Jaffee
e92203a466 add basic inspect test 2017-03-17 16:43:04 -05:00
Matt Jaffee
241649fc22 rename cmd.Serve->cmd.Server (gorename) 2017-03-17 16:43:04 -05:00
Matt Jaffee
60c1828dc9 nest profiling options in config file and test 2017-03-17 16:43:04 -05:00
Matt Jaffee
442380f8ab add anti-entropy interval arg 2017-03-17 16:43:04 -05:00
Matt Jaffee
b291307dde add plugins path arg to server
also fix an issue where tests would hang if an invalid config file was given
2017-03-17 16:43:04 -05:00
Matt Jaffee
9a18fc11a6 support cluster poll interval arg 2017-03-17 16:43:04 -05:00
Matt Jaffee
03eae77604 add cluster.hosts flag
had to work around a bunch of viper nonsense with handling string slices

changed the validator.Check to use reflect.DeepEqual

fixed a bug where env vars need to be reset between tests

had to change the structure of pilosa.config to enable the passing of hosts as a
slice of strings (it was a slice of structs which each just contained the string
host.) so we lose some generality, but we weren't using it anyway.
2017-03-17 16:43:04 -05:00
Matt Jaffee
2af42eedb5 add replicaN to pilosa server and test 2017-03-17 16:43:04 -05:00
Matt Jaffee
372b7ca1c4 full testing support for pilosa server
1. fixed a couple bugs with config file reading.
2. made cmd.Serve a global variable so tests could inspect it.
3. added table style testing for pilosa server
4. added the ability to stop server programmatically which also causes the cobra
command running it to return.
2017-03-17 16:43:04 -05:00
Matt Jaffee
076cc0960a add some proof of concept command tests 2017-03-17 16:43:04 -05:00
Matt Jaffee
961d18a3bf convert all subcommands to nonglobal style 2017-03-17 16:43:04 -05:00
Matt Jaffee
c3b7710d67 WIP on converting commands to be non-global and testable 2017-03-17 16:43:04 -05:00
Matt Jaffee
943e8ece1e add data-dir flag and simplify server/server.go
Now that the default data dir is defined by the flag, we don't need to set it
explicitly if it isn't set. We also stop reading the config file explicitly in
server.go since viper will read it - we do need to define all the config options
though before the config file will work properly.
2017-03-17 16:43:04 -05:00