Commit graph

5399 commits

Author SHA1 Message Date
Seebs
1ac00291f3 Add test for the weird remapping/cache interaction.
This test is really a test of a very specific bit of the internals
of containers_btree/containers_slice, but we can't easily test it from
there because they don't have all the logic for remapping files.

The underlying issue is that they maintain a single-item "most recent
container" cache, and this wasn't getting updated during the remap
operations, happening through containers.UpdateEvery. The fix is
probably just to make sure that UpdateEvery invalidates the cache.
2020-03-31 16:20:05 -05:00
Seebs
d26e221a91 don't call isArray on a nil *Container
differenceInPlace wasn't checking for nil containers, which are
theoretically valid empty containers. Also added a couple of other
N==0 checks to streamline the higher-level operation.
2020-03-31 16:20:05 -05:00
Cody Soyland
3b6a26e5c6
Merge pull request #219 from codysoyland/ci-updates
Add updated CircleCI config
2020-03-31 16:18:10 -05:00
Cody Soyland
3a7f385a01 Add updated CircleCI config
- Remove YAML magic
- Remove a lot of duplication
- Update linter
- Use parameterized jobs and matrix build
- Update Docker Hub CD to produce versioned and "latest" images
- Add custom shard width test to workflow
2020-03-31 15:17:11 -05:00
Travis Turner
b5aa280ced
Merge pull request #220 from travisturner/decimal-yaml
yaml marshal/unmarshal for pql.Decimal
2020-03-31 13:53:00 -05:00
Travis
7cb265098f yaml marshal/unmarshal for pql.Decimal 2020-03-31 12:43:42 -05:00
Kuba Podgórski
6dc3837c9a
WIP: fix 'unknown call: Distinct' error (#213) 2020-03-31 16:29:14 +02:00
Travis Turner
dca2120c06
Merge pull request #199 from travisturner/cluster-resize-translation-partitions
include translate partitions in cluster resize instructions
2020-03-30 22:01:09 -05:00
Kuba Podgórski
ac76f6227d
Make internal.IndexMeta.TrackExistence true 2020-03-30 21:24:16 -05:00
Travis
98c5603965
close reader. include all replias in translation partition rebalance 2020-03-30 21:24:16 -05:00
Travis
4c311aa1a7
write to temp partition file. use io.Copy 2020-03-30 21:24:16 -05:00
Travis
2724ecfd5f
WIP: include translate partitions in cluster resize instructions
This commit adds `TranslationSources` to the cluster
`ResizeInstruction`. These are the sources of translation
partitions which the receiving node needs in order to support
partition distribution in the new, resized cluster.

This also fixes a bug where index options were not being
encode in the proto Index object. That meant that the schema
transferred via protobuf was not correct. The reason why
things normally worked is because index creation typically
happens on the CreateIndex message, which does include the
options.

TODO:

- [ ] implement the TranslateStore interface for `InMemTranslateStore`
and `mock.TranslateStore`
- [ ] surely need some more tests around the `ReadFrom` and `WriteTo`
2020-03-30 21:24:16 -05:00
Travis Turner
2ad06f9423
Merge pull request #212 from travisturner/decimal-min-max-args
support pql.Decimal for decimal field min/max arguments
2020-03-30 14:32:02 -05:00
Travis
80da129861 require scale argument for decimal fields 2020-03-30 13:38:50 -05:00
Travis
1da7cf09bb support pql.Decimal for decimal field min/max arguments 2020-03-27 16:02:47 -05:00
Cory LaNou
2a4088e0ec
Merge pull request #206 from corylanou/test-helpers
Make better use of t.Helper
2020-03-25 07:35:35 -05:00
Cory LaNou
391fda9849
Merge branch 'enterprise' into test-helpers 2020-03-24 15:51:01 -05:00
corylanou
f84185230d
make use of t.Helper 2020-03-24 15:00:32 -05:00
Cody Soyland
7ba5c0e3d3
Merge pull request #201 from codysoyland/queryunary
Add QueryPQLUnary gRPC call
2020-03-23 16:58:27 -05:00
Cody Soyland
1aa1ec51aa Add QueryPQLUnary gRPC call 2020-03-23 16:24:31 -05:00
Cory LaNou
69a1365bf0
Merge pull request #202 from corylanou/sum/194
Fixed sum for negative values
2020-03-23 16:22:11 -05:00
corylanou
f32f9f64a4
fix sum for negative values 2020-03-23 14:02:46 -05:00
Travis Turner
532f746f25
Merge pull request #195 from travisturner/sync-mutex-bool
support mutex/bool fields in anti-entropy
2020-03-19 11:47:37 -05:00
Travis Turner
5cfb29beb2
Merge branch 'enterprise' into sync-mutex-bool 2020-03-19 11:28:56 -05:00
Travis Turner
564eee0bdf
Merge pull request #190 from travisturner/decimal-between
serialize decimal between pql for internode queries
2020-03-19 11:26:12 -05:00
Travis
57c30b9dae support mutex/bool fields in anti-entropy 2020-03-19 11:11:33 -05:00
Travis
4d38723fd4 serialize decimal between pql for internode queries 2020-03-19 07:50:43 -05:00
Travis Turner
419c2179b5
Merge pull request #185 from travisturner/decimal-min-max-overflow
Avoid overflow on decimal min/max default values
2020-03-18 08:01:40 -05:00
Travis
298f290e86
Avoid overflow on decimal min/max default values
If the min/max provided are already on the boundary of int64,
then we don't want to operate on them and cause overflow.
there are still overflow scenarios where a user provides a
min/max which is not on the boundary, but overflow once the
scale is applied. This does not address those cases, but at
least it addresses the default case (where a min/max is not
provided)
2020-03-18 07:41:35 -05:00
Kuba Podgórski
73ca124944
Fix runCountRange when range start == interval start (#181)
When the interval is a proper superset of the range with start equal to
interval start, the range must be considered a superset or it will be
completly ignored (since it neither a subset nor it overlaps)

Co-authored-by: Pierre Fersing <pierre.fersing@bleemeo.com>
2020-03-17 20:31:35 +01:00
Travis Turner
ba5b133e1e
Merge pull request #180 from travisturner/mu-anti-entropy
add mutex for anti-entropy and node join/leave
2020-03-17 13:05:53 -05:00
Travis
fbbd474978 add mutex for anti-entropy and node join/leave 2020-03-17 12:46:09 -05:00
Travis Turner
22ae1139d1
Merge pull request #172 from travisturner/float-to-decimal
use pql.Decimal instead of float64
2020-03-16 08:35:19 -05:00
Travis
c60241b5a9 Get rid of Sign from pql.Decimal struct
It turns out that it's not very useful to keep the sign
value as a separate argument in the pql.Decimal struct.
This commit incorporates it into Value, and makes Value
an `int64` (for some bone-headed reason I had made it a
`uint32` before which is just dumb).
2020-03-15 23:00:00 -05:00
Travis
30e08eb532 add error conditions to tests 2020-03-15 16:18:59 -05:00
Travis
513edeae9c fix min/max bug for decimal fields 2020-03-14 22:35:24 -05:00
Travis
963affcc30 WIP: use pql.Decimal instead of float64
This commit introduces a new type: pql.Decimal
We use that instead of float64 in order to ensure
that the string representation is consistent.

One unfortunate discovery during implementation is
that the RowAttrs and ColAttrs support floats, and
the PEG file was treating them as such. So I had
to split the PEG definitions into float-specific
items and decimal-specific items.
2020-03-14 22:33:52 -05:00
Travis Turner
334eb3cd08
Merge pull request #175 from travisturner/int-min-max-with-offset
Fixes the min/max bug for `int` fields with offset.
2020-03-14 15:18:48 -05:00
Travis
3863ab4b41 Fixes the min/max bug for int fields with offset.
Methods `MinForShard` and `MaxForShard` were not adjusting
their return value by the offset.
2020-03-14 14:01:40 -05:00
Travis Turner
0d55d6eab2
Merge pull request #160 from travisturner/int-fragment-sync-better-fix
support fragment sync for int and decimal fields
2020-03-13 18:42:12 -05:00
Travis
a5cba75855 fix typos in comment 2020-03-13 12:15:54 -05:00
Travis
3b676a7cbd fix linter warnings 2020-03-13 12:15:54 -05:00
Travis
cbf80370cb support fragment sync for int and decimal fields
This PR adds support for anti-entropy syncing for integer
and decimal fields. It differs from the logic for other
field types in that it does not rely on a consensus to determine
what the value should be; instead, it considers the correct
values to be those of the primary replica. From there, data
is pushed to all non-primary replicas.
2020-03-13 12:15:54 -05:00
Cody Soyland
8b5848f615
Merge pull request #164 from codysoyland/dockerfile-env-config
Use env variables instead of flags in Dockerfile
2020-03-12 14:44:45 -05:00
Cody Soyland
c6a293f818 Use env variables instead of flags in Dockerfile
This allows you to override those attributes without overriding the
entire command.
2020-03-12 11:22:31 -05:00
Travis Turner
93a804f7ab
Merge pull request #165 from travisturner/range-ast-fix
fix Call stringer for range conditional
2020-03-12 08:29:38 -05:00
Travis
0c6f0e2ba5 fix Call stringer for range conditional 2020-03-11 22:53:37 -05:00
Travis Turner
d76e8fc17c
Merge pull request #162 from travisturner/makefile-helpers
add linter and test-race targets to makefile
2020-03-11 10:10:41 -05:00
Travis
dde1e26baa add linter and test-race targets to makefile 2020-03-11 09:24:00 -05:00
Travis Turner
cdbb274a2b
Merge pull request #157 from travisturner/int-fragment-sync-quick-fix
temporary fix for int field replica sync bug
2020-03-09 15:22:41 -05:00