This mainly involved running `RecalculateCache()` on all fragments
in `TopN` tests. It seems these tests have been running as LRU
caches, and once we made `ranked` the default, they stopped working.
Some tests were affected by the change in partition number and
therefore the change in fragment to node mapping.
Previously, multiple frames with different prefixes were used to separate
different data layouts. This included separating standard row/column
layouts from inverted column/row layouts as well as storing aggregate
information for timestamp data.
Unfortunately, this caused frame meta data to be copied between multiple
frames and it made it difficult to keep these frames in sync.
This commit separates these different physical layouts into `Views`.
A `Frame` now has one or more views which represent each layout.
Fragments have been moved from under the `Frame` to be contained
within the `View`.
There are two primary views:
- `standard`
- `inverse`
If a frame has a time quantum, then views are generated for these
each of the standard/inverse views. For example a time quantum
of `YMDH` for the date `2000-01-02T00:00:00Z` would create the
following views:
- `standard_2000`
- `inverse_2000`
- `standard_200001`
- `inverse_200001`
- `standard_20000102`
- `inverse_20000102`
From the user's perspective, nothing should change in PQL. Different
PQL statements will handle the appropriate view automatically. For
example, `Bitmap()` and `Profile()` will fetch using the `standard`
view or the `inverse` view, respectively. The `Range()` statement
will lookup the appropriate time-based views automatically.
Databases and frames now require explicit creation and have the
option of setting row & column labels. If no labels are provided
then the default `id` and `profileID` labels are used.
This changes the API of the types but does not alter the functionality
since only the `context.Background()` is currently being used. Adding
`Context` will help handle fault tolerance in the future by allowing
timeouts to be propagated across calls to different nodes.
This commit adds two commands:
pilosactl backup
pilosactl restore
The `Client` implementations have also been added to support these
commands:
func (c *Client) BackupTo(w io.Writer, db, frame string) error
func (c *Client) RestoreFrom(r io.Reader, db, frame string) error
Backups occur on a per-frame basis and all slices from the cluster
are packed into a single tar file. The backup tool attempts to
read from owner nodes in a random order and will retry against
the next owner if one fails.
During restore, the slices are restored to all owner nodes. This
allows users to backup a frame from one cluster and restore it
to a different one -- even if the topology of the new cluster is
different.
