Commit graph

5459 commits

Author SHA1 Message Date
alanbernstein
0e6ed80bd4
Merge pull request #269 from alanbernstein/long-query-full
WIP Show full query string when logging slow queries
2020-04-09 16:12:11 -05:00
Alan Bernstein
92416018d5
Use more general log message 2020-04-09 15:53:40 -05:00
Alan Bernstein
5aebb242f8
Fix off-by-one error 2020-04-09 15:53:40 -05:00
Alan Bernstein
e3ace544ca
Show full query string when logging slow queries 2020-04-09 15:53:39 -05:00
tgruben
57b2f32fa1
Merge pull request #260 from tgruben/sync-start
delay start for non-coordinator
2020-04-09 14:42:41 -05:00
Todd Gruben
e7bbc3a0a8 Added delay to allow cooridinator a head start in launch on multi node clusters.
considered using Cluster.Disabled to identify if pilosa was stand alone but settled
on using an empty gossip seeds list
2020-04-09 14:20:01 -05:00
alanbernstein
bdfad719df
Merge pull request #263 from alanbernstein/grpc-multi
Support multiple dialTargets and cycle through on connection reset
2020-04-08 13:39:58 -05:00
alanbernstein
1846d21765
Merge pull request #1 from travisturner/dialtargets
cycle through dial targets regardless of error
2020-04-08 12:55:17 -05:00
Travis
8f0ae1b6f5 cycle through dial targets regardless of error 2020-04-08 12:29:15 -05:00
Alan Bernstein
adb21589f0 Support multiple dialTargets and cycle through on connection reset 2020-04-08 10:09:49 -05:00
seebs
1b5d86c8f0
Merge pull request #240 from seebs/q2perf
improve performance of difference/not
2020-04-07 20:24:51 -05:00
Seebs
be379e7806 cache AvailableShards
The computation of available shards is cheap, because realistically, virtually
no one has enough shards that the resulting bitmap is more than one container.

We don't try to fix this at the field/index levels because it's significantly
harder to do there, but I think the creation of these bitmaps is probably
the most expensive part, and switching the unions to union-in-place probably
reduces cost significantly.

Note that the bitmaps being unioned almost certainly have exactly one small
container in them.
2020-04-07 19:33:21 -05:00
Seebs
6c797e0c4e TODO => TODONE: use masks for runToBitmap
Had the code lying around from mad science elsewhere, backported.
2020-04-07 19:33:21 -05:00
Seebs
d4e496887b make differenceRunBitmap smarter
We avoid using bitmapContains so often because that turns out to be expensive.
Also, if we produce more than runMaxSize runs, we're going to convert to
a bitmap container (or possibly an array container if there were over
2048 items, but they're all singletons), and we can streamline that by just
converting the source to bitmap and returning differenceBitmapBitmap, which
is faster in this case.

This appears to overall take about half as long in the workload I was
looking at.
2020-04-07 19:33:20 -05:00
Kuba Podgórski
7868188670
Merge pull request #200 from kuba--/groupby-int/124
Group by on ints
2020-04-08 01:47:17 +02:00
Kuba Podgórski
1217deee1c Rebase 2020-04-08 01:06:18 +02:00
tgruben
64caffebe8
Merge pull request #196 from tgruben/async-available-shards
limit frequency of writes for available shards broadcast
2020-04-06 09:08:04 -05:00
Todd Gruben
6712f8cf06 removed debug log 2020-04-06 08:43:30 -05:00
Todd Gruben
5470753cb5 fix conflict 2020-04-05 18:37:11 -05:00
Travis
0e2bb550db add deleted (rebalanced) shards to remoteAvailableShards 2020-04-05 18:37:11 -05:00
Todd Gruben
6bd81b87eb unexport availableShardFileFlushDuration 2020-04-05 18:37:11 -05:00
Todd Gruben
000ea90877 unbuffer channel 2020-04-05 18:34:43 -05:00
Todd Gruben
af068e08e7 cleanup and comments 2020-04-05 18:34:43 -05:00
Todd Gruben
40a3dce93c Co-authored-by: Travis Turner <github@calfrope.com> 2020-04-05 18:34:43 -05:00
Todd Gruben
bb04f7f6ac limit frequency of writes for available shards 2020-04-05 18:31:28 -05:00
Cody Soyland
0a86f6a97b Increase gRPC maximum message length 2020-04-03 16:35:57 -05:00
Travis
b7fcfb41d9 add proto OldMin/OldMax for backward compatibility 2020-04-03 12:25:07 -05:00
Travis Turner
48879107b0
Merge pull request #244 from travisturner/fix-race
make sure frag.maxRow() call is lock-protected
2020-04-02 23:52:38 -05:00
Travis
b4f1781d30 make sure frag.maxRow() call is lock-protected 2020-04-02 23:25:57 -05:00
seebs
632f6856ab
Merge pull request #229 from seebs/checkptr
disable checkptr with go 1.14
2020-04-02 22:19:45 -05:00
seebs
a20532073b
Merge branch 'enterprise' into checkptr 2020-04-02 20:48:49 -05:00
alanbernstein
4faacad4cf
Merge pull request #242 from molecula/prometheus-options
Add options to prometheus client to support setting namespace
2020-04-02 20:24:15 -05:00
Alan Bernstein
dad4ccf103 Propagate namespace to tags client 2020-04-02 19:29:46 -05:00
Alan Bernstein
110d2b6024 Restore lost comment 2020-04-02 19:07:43 -05:00
Alan Bernstein
f15d031c55 Add options to prometheus client to support setting namespace 2020-04-02 19:05:55 -05:00
Travis Turner
7432546af8
Merge pull request #241 from travisturner/shardwidth22-tests
alter tests to allow for shardwidth22
2020-04-02 18:25:29 -05:00
Travis
182c1d3c42 alter tests to allow for shardwidth22
also, reset BitDepth on field and bsiGroup during
importRoaringOverwrite
2020-04-02 17:32:49 -05:00
Cody Soyland
da503776e2
Merge pull request #237 from codysoyland/ci-tweaks
CI: Modify Docker Hub rules and use "make test-race" for running race detector
2020-04-02 12:25:40 -05:00
Cody Soyland
6a639efdba Modify Docker Hub deployment filter rules.
The documentation is unclear/incorrect, and these filters aren't
behaving correctly. This is an attempt at fixing that. More info at:
https://discuss.circleci.com/t/job-runs-even-when-tags-ignore-filter-is-triggered-when-combined-with-branches-only/20664/11
2020-04-02 12:14:37 -05:00
Cody Soyland
14319cc54b Use "make test-race" instead of custom test flags in CI 2020-04-02 12:14:37 -05:00
Seebs
c6799bd604 disable checkptr with go 1.14
The checkptr feature is actually probably right about a few
things in roaring and boltdb, but we can ignore them for now, and
that prevents checking for races, so we disable that temporarily.

Also supply NOCHECKPTR in non-race tests because CI uses "make test"
with -race in $TESTFLAGS and we might do that on other occasions.
2020-04-02 11:37:06 -05:00
Matthew Jaffee
44fcbee447
Merge pull request #235 from jaffee/nil-check-indexmeta
add nil check for index meta
2020-04-02 11:33:25 -05:00
Matt Jaffee
35fa26918d
add nil check for index meta 2020-04-02 11:09:07 -05:00
Travis Turner
794bc5b168
Merge pull request #232 from travisturner/decimal-grcp
convert grpc response to use pql.Decimal
2020-04-02 07:55:26 -05:00
Travis
4d985653ae convert grpc response to use pql.Decimal 2020-04-01 23:59:50 -05:00
Travis Turner
0b480f8405
Merge pull request #231 from travisturner/reintroduce-decimal
Reintroduce decimal
2020-04-01 21:44:26 -05:00
Travis
22cca67d6a Revert "back out the pql.Decimal changes"
This reverts commit 741ba9b268.
2020-04-01 17:46:46 -05:00
Matthew Jaffee
e77bf45643
Merge pull request #227 from travisturner/backout-decimal
back out the pql.Decimal changes
2020-04-01 11:32:02 -05:00
Travis
741ba9b268 back out the pql.Decimal changes 2020-04-01 11:10:33 -05:00
Cody Soyland
1284791d22
Merge pull request #224 from codysoyland/ci-fixes
CI fixes: quote TESTFLAGS and fix stable release filter
2020-04-01 09:41:38 -05:00