We execute the aggregate Distinct calls after the GroupBy is complete,
and we need these to act like non-remote calls in that they forward to
all nodes, but like remote calls in that they bypass key
translation. Added a "PreTranslated" flag to the QueryRequest to
achieve this.
Discovered an issue where a nil *Row in EmbeddedData would cause a
panic in the protobuf serialization. Changed the encoding code we
control to never pass a nil *Row.
Got fed up with lack of context on errors and added wrapping to all
calls under executor.executeCall as well as a few other places.
Handled a situation where not having data on a shard for a particular
field could cause a query to error instead of just treating that
fragment as being empty. (see the switch in executeDistinctShardSet)
Stopped GroupBy from executing the Count(Distinct) aggregate on Remote
calls.
Fixed a longstanding issue where errors retrieved from remote query
calls had a garbage character at the front due to treating a protobuf
payload as an error message instead of decoding it. (see
http/client.go)
this should avoid a race condition with CreateField where createdAt
can get out of sync if there are multiple concurrent requests.
The client methods didn't allow specification of the URI, so I
modified the implementation to find the coordinator and send to it
explicitly.
- default Tx is once again RBF, changed from bolt.
- document the RBF code review comments that were not addressed
before #1052 was merged, so they don't get lost.
- they should be easily addressed by replaying the entire WAL file
rather than from the DB meta page 0 notion of the last WalID
- cleanup rbf/cfg/cfg.go stale comments, ensure default0 respected.
1 msec checkpoint time, 1MB wal segment defaults.
- return a specific error, ErrNoMetaFound, from findNextWALMetaPage()
rather than io.EOF, since there actually wasn't any file IO involved.
- add http handlers for /cpu-profile/start and /cpu-profile/stop
in http/handler.go enable CPU profiling at specific time points
during an ingest or other operation.
Add an exported IntersectionAny() from roaring to let us quickly
check whether two containers have overlap, so we can avoid performing
intersections we don't need to when evaluating containers within
the same row as a previous match. (IntersectionCount on the whole
bitmap would imply doing up to 16 intersections even if we find a bit
right away.)
We also allow ForeignIndex to be set on set, mutex, and time fields,
since all of those could now be reasonable operands for Distinct
ops.
Not yet present: Handling time quantums, but that seems really
desireable.
This is no longer necessary, as caches now recalculate on read.
Also, in general a user will not explicitly request recalculation, so it would make sense for our tests to reflect that.
- introduce Query Context (Qcx) for managing database-per-shard.
- replaces the MultiTx, so mtx.go is retired and removed.
- introduces the HolderConfig struct and all Holders now have
a path from birth.
- rbf speedups on bitwise writes
- badgerdb is removed due to unresolvable write conflicts.
fixes#703#676
