Commit graph

16 commits

Author SHA1 Message Date
Ben Johnson
3a43a862ec
Add Holder test coverage. 2017-06-12 13:07:26 -06:00
Michael Baird
60cd24b368 set maxSlice StatsD gauge on get max slice 2017-05-16 14:00:54 -05:00
Michael Baird
4a051900c0 Add sampling rate to StatsD functions, and fix data dog logging 2017-05-16 10:35:49 -05:00
Matt Jaffee
11b84cec94 change default cache type to ranked
All tests which use MustOpenFragment now explicitly pass a cacheType parameter.
If this parameter is an empty string, it means that whether the test works
should not depend on the cache type of the fragment. Otherwise the test should
explicitly set the cache type it needs rather than relying on the default.
2017-05-02 16:58:03 -05:00
Cody Soyland
3996b56b88 Apply Apache License 2.0 2017-04-28 14:22:03 -05:00
Travis
7246e1e00d
Broadcast CreateSliceMessage when a new max slice is created 2017-04-27 11:44:13 -05:00
Travis
2ad322c2c7
Rename DB to Index
Rename `db.go` to `index.go` and `db_test.go` to `index_test.go`
2017-04-23 21:49:27 -05:00
Ben Johnson
d34687121f
Rename bitmap/profile to row/column. 2017-04-21 21:23:24 -06:00
Travis
25e649add8 remove cacheSize argument from NewFragment() 2017-04-19 11:41:41 -05:00
Travis
b0f1fc7523 Makes Messenger a first-class object under Server (with pointers in Handler and Index).
Primary message interface is the MessageBroker which is an attribute of the Messenger.
MessageBroker implementations:
- Gossip (memberlist)
- Broadcast (uses HTTP, received by existing Handler)
- Static (no-ops)

Changes CacheSize from `int` to `uint32` for consistency with protobuf.
Removes unnecessary dependencies in glide:
- `github.com/aws/aws-sdk-go`
- `golang.org/x/net` (although this gets included by memberlist)

TODO:
- [ ] Add tests around the Messenger and MessageBroker objects.
- [ ] Refactor CreateSliceMessage to work with views.
- [ ] Support propogation of meta data on PATCH calls.

fixing some issues from last rebase
2017-04-19 10:33:26 -05:00
Michael Baird
8ceaef1eaa fragment cache size is set on frame creation with a default value of 50,000 2017-04-19 10:06:10 -05:00
Ben Johnson
d32edb5329 Add cacheType to frame options.
Previously the cache type (`lru` or `ranked`) was determined by
checking the ending suffix of the frame. If it ended with a `.n`
then it was ranked. Otherwise it was an LRU.

This commit changes frames so that a `cacheType` option can be
passed in during creation to set either `lru` or `ranked`.
2017-04-18 08:24:58 -06:00
Ben Johnson
306cc0b64e Opt-in to inverse storage. 2017-04-12 08:34:31 -06:00
Ben Johnson
2731311378 Refactor higher level view interface. 2017-04-05 17:00:31 -06:00
Ben Johnson
3a71003240 Re-sort inverted bits on import. 2017-03-24 13:57:02 -06:00
Ben Johnson
eff08af112 Separate physical data layout with views.
Previously, multiple frames with different prefixes were used to separate
different data layouts. This included separating standard row/column
layouts from inverted column/row layouts as well as storing aggregate
information for timestamp data.

Unfortunately, this caused frame meta data to be copied between multiple
frames and it made it difficult to keep these frames in sync.

This commit separates these different physical layouts into `Views`.
A `Frame` now has one or more views which represent each layout.
Fragments have been moved from under the `Frame` to be contained
within the `View`.

There are two primary views:

- `standard`
- `inverse`

If a frame has a time quantum, then views are generated for these
each of the standard/inverse views. For example a time quantum
of `YMDH` for the date `2000-01-02T00:00:00Z` would create the
following views:

- `standard_2000`
- `inverse_2000`
- `standard_200001`
- `inverse_200001`
- `standard_20000102`
- `inverse_20000102`

From the user's perspective, nothing should change in PQL. Different
PQL statements will handle the appropriate view automatically. For
example, `Bitmap()` and `Profile()` will fetch using the `standard`
view or the `inverse` view, respectively. The `Range()` statement
will lookup the appropriate time-based views automatically.
2017-03-24 13:57:02 -06:00