This mainly involved running `RecalculateCache()` on all fragments
in `TopN` tests. It seems these tests have been running as LRU
caches, and once we made `ranked` the default, they stopped working.
Some tests were affected by the change in partition number and
therefore the change in fragment to node mapping.
Primary message interface is the MessageBroker which is an attribute of the Messenger.
MessageBroker implementations:
- Gossip (memberlist)
- Broadcast (uses HTTP, received by existing Handler)
- Static (no-ops)
Changes CacheSize from `int` to `uint32` for consistency with protobuf.
Removes unnecessary dependencies in glide:
- `github.com/aws/aws-sdk-go`
- `golang.org/x/net` (although this gets included by memberlist)
TODO:
- [ ] Add tests around the Messenger and MessageBroker objects.
- [ ] Refactor CreateSliceMessage to work with views.
- [ ] Support propogation of meta data on PATCH calls.
fixing some issues from last rebase
this affects the scanner/parser and makes it produce signed ints rather than
uints, and so it affects all code that is expecting unsighed ints to come out of
the parser.
Previously only the standard max slice was available. This commit
changes it so that the inverse max can be retrieved separately
through the `DB` and `Frame` types as well as through the
`HTTP` API and `Client`.
Previously, multiple frames with different prefixes were used to separate
different data layouts. This included separating standard row/column
layouts from inverted column/row layouts as well as storing aggregate
information for timestamp data.
Unfortunately, this caused frame meta data to be copied between multiple
frames and it made it difficult to keep these frames in sync.
This commit separates these different physical layouts into `Views`.
A `Frame` now has one or more views which represent each layout.
Fragments have been moved from under the `Frame` to be contained
within the `View`.
There are two primary views:
- `standard`
- `inverse`
If a frame has a time quantum, then views are generated for these
each of the standard/inverse views. For example a time quantum
of `YMDH` for the date `2000-01-02T00:00:00Z` would create the
following views:
- `standard_2000`
- `inverse_2000`
- `standard_200001`
- `inverse_200001`
- `standard_20000102`
- `inverse_20000102`
From the user's perspective, nothing should change in PQL. Different
PQL statements will handle the appropriate view automatically. For
example, `Bitmap()` and `Profile()` will fetch using the `standard`
view or the `inverse` view, respectively. The `Range()` statement
will lookup the appropriate time-based views automatically.
Databases and frames now require explicit creation and have the
option of setting row & column labels. If no labels are provided
then the default `id` and `profileID` labels are used.
Consolidates all the pql.Call implementations into a single
pql.Call struct. This is needed to support user-defined schemas
on frames.
This also has the added benefit that a lot of redundant parsing
code has been consolidated.
This changes the API of the types but does not alter the functionality
since only the `context.Background()` is currently being used. Adding
`Context` will help handle fault tolerance in the future by allowing
timeouts to be propagated across calls to different nodes.
This commit refactors the block checksumming by removing the
iteration over each block and instead only checking blocks which
have data. This requires merging the cache inspection with the
roaring iterator to reduce CPU time and memory allocations.
