Commit graph

8201 commits

Author SHA1 Message Date
Souhaila Noor
642d4fcb86 fixed variable name 2021-11-05 13:27:03 -05:00
Seebs
c7e4bc0fd0 refactoring experiment
I'm not actually sold on this, but I'm not entirely unsold on it. It seems like
it does reduce the amount of duplication a lot, but also it's sort of a mess.

In the process, noticed that it makes more sense to grab the whole cluster
rather than just the nodes for an arbitrary shard for the shard==^0 case,
because then if we have an API (but no Qcx), we can be reasonably confident
that we'll be able to pick the local node for loopback even if we aren't
using the API directly.

Have thought about whether we should create our own Qcx in cases like that
but I really don't like the idea of automatically creating a Qcx.
2021-11-05 13:06:38 -05:00
Seebs
d4b06d077e Import/ImportValue API rework and improvements
Underlying goal: Don't use the http client to send messages back to the
local host. Also, when sending data to other nodes, don't collate it
from an ImportRequest into a completely different format, then immediately
collate that back into an ImportRequest. This does require changing
the logic over in ctl/import to make it create an ImportRequest.

Also, add additional testing to make sure we're actually trying anything
at all with several combinations (such as submitting import requests
which don't match the configuration of index or field), and improve
test coverage for that.

This introduces the ability to tell an http/client InternalClient about
a specific API that it should use for local queries where applicable.
That's not implemented outside of the import stuff, but should probably
be applied eventually to other things that are trying to talk to many
nodes one of which may be the local node. That behavior is contingent
on passing in a Qcx, because it is implicitly tied to an existing
execution context, and it can't assume that it can create a new one,
because that could deadlock.
2021-11-05 13:06:38 -05:00
Seebs
60ac6a929f make address lookup failures okay on MacOS 2021-11-05 13:06:38 -05:00
Seebs
cff8da346f Improvements and benchmarks for ingest field operation sorting
Reusing the field operation sorting for other things caused me
to hit a bug, also made me curious about a performance issue and
whether it was possible to improve it. Answer: Not easy to improve,
anyway.
2021-11-05 13:06:38 -05:00
Souhaila Noor
4f964aeef4 updated ssh config 2021-11-05 12:53:39 -05:00
Souhaila Noor
86407ebcab sanity check for scp 2021-11-05 12:39:11 -05:00
Souhaila Noor
7c02e9c447 added scp 2021-11-05 12:37:15 -05:00
Ben Johnson
f2bc887d6f
Merge pull request #1751 from molecula/fb-904
[FB-904] Add deb/rpm packaging of FeatureBase binary
2021-11-05 10:59:48 -06:00
Souhaila Noor
d080164a49 echo aws config 2021-11-05 11:35:54 -05:00
Souhaila Noor
ab3679bef7 sanity check 2021-11-05 11:33:17 -05:00
Souhaila Noor
6e6d7a77db debug echo statements 2021-11-05 11:26:09 -05:00
tgruben
fd08815f51
Merge branch 'master' into fb-904 2021-11-05 11:23:01 -05:00
Souhaila Noor
a9ab597d28 added profile to start-instance 2021-11-05 11:20:21 -05:00
Souhaila Noor
021af2c5e5 changed ssh key name to match aws 2021-11-05 10:56:10 -05:00
Souhaila Noor
0cfd5f116d echo ssh file 2021-11-05 10:50:59 -05:00
Souhaila Noor
550a610b5b sanity checks 2021-11-05 10:46:54 -05:00
Souhaila Noor
f7497bfe8f pass ssh key to a file 2021-11-05 10:35:15 -05:00
Souhaila Noor
6643e3963e reverted back to variable for ssh 2021-11-05 10:28:36 -05:00
Souhaila Noor
04b64c0f30 add path for ssh keys 2021-11-05 09:54:37 -05:00
Samir Patel
6fa8f0a88e
Merge pull request #1749 from molecula/bug/index-bounds
[FB-965] featurebase import: check for data in requests
2021-11-05 09:49:50 -05:00
Souhaila Noor
bd440f4c23 updated permissions for ssh key 2021-11-05 09:43:19 -05:00
Souhaila Noor
d147c03e22 add ssh config in section with using ssh keys 2021-11-05 09:32:53 -05:00
Souhaila Noor
7271821ebe fixed path for ssh keys 2021-11-05 09:28:25 -05:00
Souhaila Noor
14a9662494 fixed yaml syntax error 2021-11-05 09:24:20 -05:00
Souhaila Noor
f9ff3973d0 fixed argument name 2021-11-05 09:17:19 -05:00
Samir Patel
139f4c9cbb
Merge branch 'master' into bug/index-bounds 2021-11-05 09:16:39 -05:00
Samir Patel
cb6a364890 more qcx defers 2021-11-05 09:12:28 -05:00
Souhaila Noor
383663673b configure ssh 2021-11-05 09:11:27 -05:00
Souhaila Noor
321d273dce fixed typo in instance name 2021-11-05 09:03:51 -05:00
Ben Johnson
94c6c4ee53
Merge branch 'master' into fb-904 2021-11-04 16:58:30 -06:00
Samir Patel
20131a697c add additional tests and defer qcx 2021-11-04 17:07:24 -05:00
Samir Patel
cdaec2e0d2 remove duplicate checks 2021-11-04 16:04:00 -05:00
Samir Patel
67233d5720 move checks to api.go 2021-11-04 15:52:37 -05:00
Souhaila Noor
82b97022ff removed sleep time 2021-11-04 14:57:41 -05:00
souhailanoor
268ea74856
Merge branch 'master' into FB-900_singleNodeDeployment 2021-11-04 14:47:46 -05:00
Souhaila Noor
f80f2fc702 added rsa key 2021-11-04 14:34:19 -05:00
Souhaila Noor
08161a366a undo last commit 2021-11-04 13:42:03 -05:00
Souhaila Noor
20224b1df5 updated aws config 2021-11-04 13:28:48 -05:00
reese
2e9b43dfca
Merge pull request #1752 from molecula/sup-86
[SUP-86] slow webui on large query
2021-11-04 12:54:11 -05:00
Samir Patel
c0e201d638 dedup api_test 2021-11-04 12:19:56 -05:00
reesporte
f70fdfb59e remove inline styling 2021-11-04 12:14:01 -05:00
Souhaila Noor
8274a32127 debug profile 2021-11-04 12:09:20 -05:00
reesporte
4891cd1c9a Merge branch 'sup-86' of github.com:molecula/featurebase into sup-86 2021-11-04 11:55:59 -05:00
reesporte
7b3dee5653 single quotify 2021-11-04 11:55:32 -05:00
Souhaila Noor
7a6e7c2fd5 set profile 2021-11-04 11:53:43 -05:00
Souhaila Noor
01260cfcce update profile 2021-11-04 11:38:58 -05:00
Ben Johnson
395f35fc3d FB-904: Add deb/rpm packaging of FeatureBase binary 2021-11-04 10:32:59 -06:00
Souhaila Noor
502f40593e updated profile 2021-11-04 11:26:05 -05:00
Souhaila Noor
67da6d9ffa debug profile 2021-11-04 11:12:30 -05:00