Commit graph

13 commits

Author SHA1 Message Date
Ben Johnson
7ff684c193
Allow partial translate file reads.
This commit fixes an issue where translation `LogEntry` must be
read in its entirety, however, large entries can exceed the buffer
size. This has been changed so that partial entries reads are allowed.

The `LogEntry.ReadFrom()` may still generate large byte slices
during reads of large individual fields or keys.
2019-06-04 10:08:52 -06:00
kuba--
51ac675e82
TranslateFile - reopen the same instance
Signed-off-by: kuba-- <kuba@sourced.tech>
2019-05-02 14:05:40 +02:00
Cody Soyland
fdbfc68f7c Add license headers to files missing them and CI check to verify they are present. Fixes #1633 2019-04-12 11:30:41 -05:00
Matt Jaffee
fe7b926773
make sure more tests and benchmarks can have their temp dir set by flag
This is to allow the directory to be set to where a particular disk is mounted
during benchmarking.
2019-01-21 16:23:27 -06:00
Travis Turner
08d7f65667
increase the translate file size for tests/benchmarks 2018-11-16 13:01:21 -06:00
Cody Soyland
2394d36108 Adjust tests, fix 32-bit config 2018-09-19 13:30:22 -05:00
Travis Turner
ceaf97de60
add translate tests to ensure that adding/removing replica nodes behaves as expected 2018-08-13 11:06:16 -05:00
Travis Turner
b304de6536
Treat coordinator as primary translate store.
Daisy-chain other nodes based on their position in the cluster.
Deprecate the `primary-url` configuration option.
2018-08-13 11:06:16 -05:00
Cody Soyland
dd7b5f517b
Merge branch 'master' into 1513-metalinter-megacheck 2018-08-02 10:29:47 -05:00
Travis Turner
3ffafaca4a
ignore remote translation 2018-07-30 11:45:34 -05:00
Cody Soyland
f01d850b17 Fix linter issues: gosimple 2018-07-20 09:06:43 -05:00
Matt Jaffee
68f341ea9f
fix race cond in translate_test
implements locked Reader method on test.TranslateFile. This race condition is
only present in tests, and is due to a replica referring directly to a primary
instead of through an http.TranslateStore as it would in production. This patch
requires the replica to obtain a lock while accessing the primary, and requires
test.Reopen to obtain a lock when swapping out the pilosa.TranslateStore.

I was able to reproduce the race by running: "go test
-run=TestTranslateFile_PrimaryTranslateStore -race -count=10", and could not
reproduce it after this patch.

Somewhat unrelated, I came across a "fatal error: fault" triggered by trying to
print an open pilosa.TranslateFile. This seems to be related to the mmapped
".data" field. I wrote the test to document the issue, but I don't think it's
easily fixable.
2018-07-19 12:06:31 -05:00
Ben Johnson
060254e0d6
Key-to-ID Translation
This commit adds id-to-key translation to make it easier for users
to provide non-integer identifiers for rows & columns.
2018-06-15 16:45:05 -06:00