This to decouple the implementation of recalculating caches from the
structure of the hierarchy: holder > index > frame > view > fragment.
See [Law of Demeter](https://en.wikipedia.org/wiki/Law_of_Demeter).
The PQL looks like:
```
Range(frame=f, field0 >< [200,610])
```
One thing I noticed while implementing this is that it doesn't seem
like `FieldRange()` is used in either `Frame` or `View`; the Executor
calls `Fragment.FieldRange()` directly. The problem with this is that
the offset logic is calculated in the Frame, but since the Executor
doesn't go through Frame, then the Executor also has to calculate
the offset before calling `Fragment.FieldRange`. We should unify this
logic somewhere. Note, this applies to both `FieldRange` and
`FieldRangeBetween`.
This required changing the Holder test utility Reopen, since in those cases we needed to close the file before making permission changes that Close() was failing silently on.
This commit adds `FieldRange()` to the `Frame`, `View`, and
`Fragment` types. It accepts an operator & a predicate and
returns a bitmap of matching field values.
All tests which use MustOpenFragment now explicitly pass a cacheType parameter.
If this parameter is an empty string, it means that whether the test works
should not depend on the cache type of the fragment. Otherwise the test should
explicitly set the cache type it needs rather than relying on the default.
Primary message interface is the MessageBroker which is an attribute of the Messenger.
MessageBroker implementations:
- Gossip (memberlist)
- Broadcast (uses HTTP, received by existing Handler)
- Static (no-ops)
Changes CacheSize from `int` to `uint32` for consistency with protobuf.
Removes unnecessary dependencies in glide:
- `github.com/aws/aws-sdk-go`
- `golang.org/x/net` (although this gets included by memberlist)
TODO:
- [ ] Add tests around the Messenger and MessageBroker objects.
- [ ] Refactor CreateSliceMessage to work with views.
- [ ] Support propogation of meta data on PATCH calls.
fixing some issues from last rebase
Previously the cache type (`lru` or `ranked`) was determined by
checking the ending suffix of the frame. If it ended with a `.n`
then it was ranked. Otherwise it was an LRU.
This commit changes frames so that a `cacheType` option can be
passed in during creation to set either `lru` or `ranked`.
Previously, multiple frames with different prefixes were used to separate
different data layouts. This included separating standard row/column
layouts from inverted column/row layouts as well as storing aggregate
information for timestamp data.
Unfortunately, this caused frame meta data to be copied between multiple
frames and it made it difficult to keep these frames in sync.
This commit separates these different physical layouts into `Views`.
A `Frame` now has one or more views which represent each layout.
Fragments have been moved from under the `Frame` to be contained
within the `View`.
There are two primary views:
- `standard`
- `inverse`
If a frame has a time quantum, then views are generated for these
each of the standard/inverse views. For example a time quantum
of `YMDH` for the date `2000-01-02T00:00:00Z` would create the
following views:
- `standard_2000`
- `inverse_2000`
- `standard_200001`
- `inverse_200001`
- `standard_20000102`
- `inverse_20000102`
From the user's perspective, nothing should change in PQL. Different
PQL statements will handle the appropriate view automatically. For
example, `Bitmap()` and `Profile()` will fetch using the `standard`
view or the `inverse` view, respectively. The `Range()` statement
will lookup the appropriate time-based views automatically.
