Commit graph

8 commits

Author SHA1 Message Date
Travis
25e649add8 remove cacheSize argument from NewFragment() 2017-04-19 11:41:41 -05:00
Travis
b0f1fc7523 Makes Messenger a first-class object under Server (with pointers in Handler and Index).
Primary message interface is the MessageBroker which is an attribute of the Messenger.
MessageBroker implementations:
- Gossip (memberlist)
- Broadcast (uses HTTP, received by existing Handler)
- Static (no-ops)

Changes CacheSize from `int` to `uint32` for consistency with protobuf.
Removes unnecessary dependencies in glide:
- `github.com/aws/aws-sdk-go`
- `golang.org/x/net` (although this gets included by memberlist)

TODO:
- [ ] Add tests around the Messenger and MessageBroker objects.
- [ ] Refactor CreateSliceMessage to work with views.
- [ ] Support propogation of meta data on PATCH calls.

fixing some issues from last rebase
2017-04-19 10:33:26 -05:00
Michael Baird
8ceaef1eaa fragment cache size is set on frame creation with a default value of 50,000 2017-04-19 10:06:10 -05:00
Ben Johnson
d32edb5329 Add cacheType to frame options.
Previously the cache type (`lru` or `ranked`) was determined by
checking the ending suffix of the frame. If it ended with a `.n`
then it was ranked. Otherwise it was an LRU.

This commit changes frames so that a `cacheType` option can be
passed in during creation to set either `lru` or `ranked`.
2017-04-18 08:24:58 -06:00
Ben Johnson
306cc0b64e Opt-in to inverse storage. 2017-04-12 08:34:31 -06:00
Ben Johnson
2731311378 Refactor higher level view interface. 2017-04-05 17:00:31 -06:00
Ben Johnson
3a71003240 Re-sort inverted bits on import. 2017-03-24 13:57:02 -06:00
Ben Johnson
eff08af112 Separate physical data layout with views.
Previously, multiple frames with different prefixes were used to separate
different data layouts. This included separating standard row/column
layouts from inverted column/row layouts as well as storing aggregate
information for timestamp data.

Unfortunately, this caused frame meta data to be copied between multiple
frames and it made it difficult to keep these frames in sync.

This commit separates these different physical layouts into `Views`.
A `Frame` now has one or more views which represent each layout.
Fragments have been moved from under the `Frame` to be contained
within the `View`.

There are two primary views:

- `standard`
- `inverse`

If a frame has a time quantum, then views are generated for these
each of the standard/inverse views. For example a time quantum
of `YMDH` for the date `2000-01-02T00:00:00Z` would create the
following views:

- `standard_2000`
- `inverse_2000`
- `standard_200001`
- `inverse_200001`
- `standard_20000102`
- `inverse_20000102`

From the user's perspective, nothing should change in PQL. Different
PQL statements will handle the appropriate view automatically. For
example, `Bitmap()` and `Profile()` will fetch using the `standard`
view or the `inverse` view, respectively. The `Range()` statement
will lookup the appropriate time-based views automatically.
2017-03-24 13:57:02 -06:00