Commit graph

8007 commits

Author SHA1 Message Date
Seebs
c7e4bc0fd0 refactoring experiment
I'm not actually sold on this, but I'm not entirely unsold on it. It seems like
it does reduce the amount of duplication a lot, but also it's sort of a mess.

In the process, noticed that it makes more sense to grab the whole cluster
rather than just the nodes for an arbitrary shard for the shard==^0 case,
because then if we have an API (but no Qcx), we can be reasonably confident
that we'll be able to pick the local node for loopback even if we aren't
using the API directly.

Have thought about whether we should create our own Qcx in cases like that
but I really don't like the idea of automatically creating a Qcx.
2021-11-05 13:06:38 -05:00
Seebs
d4b06d077e Import/ImportValue API rework and improvements
Underlying goal: Don't use the http client to send messages back to the
local host. Also, when sending data to other nodes, don't collate it
from an ImportRequest into a completely different format, then immediately
collate that back into an ImportRequest. This does require changing
the logic over in ctl/import to make it create an ImportRequest.

Also, add additional testing to make sure we're actually trying anything
at all with several combinations (such as submitting import requests
which don't match the configuration of index or field), and improve
test coverage for that.

This introduces the ability to tell an http/client InternalClient about
a specific API that it should use for local queries where applicable.
That's not implemented outside of the import stuff, but should probably
be applied eventually to other things that are trying to talk to many
nodes one of which may be the local node. That behavior is contingent
on passing in a Qcx, because it is implicitly tied to an existing
execution context, and it can't assume that it can create a new one,
because that could deadlock.
2021-11-05 13:06:38 -05:00
Seebs
60ac6a929f make address lookup failures okay on MacOS 2021-11-05 13:06:38 -05:00
Seebs
cff8da346f Improvements and benchmarks for ingest field operation sorting
Reusing the field operation sorting for other things caused me
to hit a bug, also made me curious about a performance issue and
whether it was possible to improve it. Answer: Not easy to improve,
anyway.
2021-11-05 13:06:38 -05:00
Ben Johnson
f2bc887d6f
Merge pull request #1751 from molecula/fb-904
[FB-904] Add deb/rpm packaging of FeatureBase binary
2021-11-05 10:59:48 -06:00
tgruben
fd08815f51
Merge branch 'master' into fb-904 2021-11-05 11:23:01 -05:00
Samir Patel
6fa8f0a88e
Merge pull request #1749 from molecula/bug/index-bounds
[FB-965] featurebase import: check for data in requests
2021-11-05 09:49:50 -05:00
Samir Patel
139f4c9cbb
Merge branch 'master' into bug/index-bounds 2021-11-05 09:16:39 -05:00
Samir Patel
cb6a364890 more qcx defers 2021-11-05 09:12:28 -05:00
Ben Johnson
94c6c4ee53
Merge branch 'master' into fb-904 2021-11-04 16:58:30 -06:00
Samir Patel
20131a697c add additional tests and defer qcx 2021-11-04 17:07:24 -05:00
Samir Patel
cdaec2e0d2 remove duplicate checks 2021-11-04 16:04:00 -05:00
Samir Patel
67233d5720 move checks to api.go 2021-11-04 15:52:37 -05:00
reese
2e9b43dfca
Merge pull request #1752 from molecula/sup-86
[SUP-86] slow webui on large query
2021-11-04 12:54:11 -05:00
Samir Patel
c0e201d638 dedup api_test 2021-11-04 12:19:56 -05:00
reesporte
f70fdfb59e remove inline styling 2021-11-04 12:14:01 -05:00
reesporte
4891cd1c9a Merge branch 'sup-86' of github.com:molecula/featurebase into sup-86 2021-11-04 11:55:59 -05:00
reesporte
7b3dee5653 single quotify 2021-11-04 11:55:32 -05:00
Ben Johnson
395f35fc3d FB-904: Add deb/rpm packaging of FeatureBase binary 2021-11-04 10:32:59 -06:00
reese
cdbd8b6da7
Merge branch 'master' into sup-86 2021-11-03 13:20:21 -05:00
reesporte
e11ce9a248 clearer wording 2021-11-03 13:18:41 -05:00
reesporte
1585313a0c better styling 2021-11-03 13:14:28 -05:00
reesporte
f9c78091ab move warning label to results count 2021-11-03 13:05:47 -05:00
tgruben
63fd5100d1
Merge pull request #1740 from molecula/golangci-lint
[FB-895] gofmt govet
2021-11-03 10:14:47 -05:00
Todd Gruben
2e086b866a . 2021-11-03 09:37:52 -05:00
Todd Gruben
792032d8b2 Merge branch 'golangci-lint' of github.com:molecula/pilosa into golangci-lint 2021-11-03 09:02:56 -05:00
Todd Gruben
41ed706d83 add license 2021-11-03 09:02:47 -05:00
tgruben
3318b79710
Merge branch 'master' into golangci-lint 2021-11-03 08:58:33 -05:00
Todd Gruben
7fa71d9548 check selecthandler 2021-11-03 08:57:24 -05:00
Todd Gruben
e915d75df6 remove @ from yaml
try to fix yml syntax

same

same

same

same2

same3

same4

same5

try with shell runner instead of dind

remove lattice from dockerfile

change path to bin

runs after linux arm64 build

change dockerfile path

same

same

add dir

better test coverage
2021-11-03 08:48:57 -05:00
reesporte
ad2f5c5ed4 limit the number of results that we keep
also tells the user how many results there were, but they only
get to see 1000 of them
2021-11-02 16:03:43 -05:00
Samir Patel
2bc2640fbf test two scenarios 2021-11-02 15:50:48 -05:00
Samir Patel
3ca91bba7f More specific error message on mismatch 2021-11-02 15:49:59 -05:00
Todd Gruben
a79bb893af test ToRows 2021-11-02 12:14:57 -05:00
Todd Gruben
f8d4d83e1a check op log 2021-11-02 10:54:56 -05:00
Samir Patel
4fd2a4fedd
Merge pull request #1737 from molecula/CI/build-containers
[CORE-898] Create FeatureBase Docker Images in CI
2021-11-02 09:41:24 -05:00
reesporte
5d7a399500 go mod tidy! 2021-11-01 19:24:18 -05:00
Samir Patel
5206e8fa0e more comprehensive value check 2021-11-01 18:19:38 -05:00
Samir Patel
1de6085782 remove empty value check from handler 2021-11-01 17:51:16 -05:00
Samir Patel
002b65c516 add checks for data presence on import req 2021-11-01 17:04:33 -05:00
reesporte
25d9895a21 actually get coverage on every single package and subpackage (slow) 2021-11-01 16:49:27 -05:00
reesporte
35d35d9d4d get rid of coverage html since we dont use it 2021-11-01 14:33:57 -05:00
Todd Gruben
2f92fd2a9a all the things 2021-11-01 12:49:27 -05:00
Todd Gruben
ff6a85b728 cvrpkg try again 2021-11-01 12:33:59 -05:00
Todd Gruben
745c34bbfd remove syntload 2021-11-01 12:27:44 -05:00
Todd Gruben
b9b9b0b9cf coverage fun part2 2021-11-01 12:17:17 -05:00
Todd Gruben
76f7690ede Merge branch 'golangci-lint' of github.com:molecula/pilosa into golangci-lint 2021-11-01 11:59:45 -05:00
Todd Gruben
2dab5ac8b2 coverage fun 2021-11-01 11:59:22 -05:00
tgruben
7e0ada9e1a
Merge branch 'master' into golangci-lint 2021-11-01 11:33:56 -05:00
Todd Gruben
9f67866e95 include subpackages in coverage 2021-11-01 11:21:57 -05:00