Commit graph

8920 commits

Author SHA1 Message Date
Souhaila Noor
dfc98e3000 updated sleep time 2021-11-07 09:11:52 -06:00
Souhaila Noor
57d4eb45de gitlab path test2 2021-11-07 08:53:01 -06:00
Souhaila Noor
0e564a00e8 path for cloud-init 2021-11-07 08:47:10 -06:00
Souhaila Noor
606b718428 fixed path for cloud-init 2021-11-07 08:42:40 -06:00
Souhaila Noor
78713c6a8c test cloudinit 2021-11-07 08:39:43 -06:00
reesporte
66c7d59707 remove dot imports from catcher.go 2021-11-05 15:39:50 -05:00
Souhaila Noor
103312be8f added cloutinit 2021-11-05 15:04:09 -05:00
Matthew Jaffee
8d47efb666
Merge pull request #1753 from molecula/add-fgprof
add fgprof endpoint by default
2021-11-05 14:42:18 -05:00
Souhaila Noor
6531f90747 fixed syntax error 2021-11-05 14:26:20 -05:00
Souhaila Noor
bf93278ca3 scp binary 2021-11-05 14:24:02 -05:00
Souhaila Noor
179a0e7624 fixed typo for copy path 2021-11-05 14:03:14 -05:00
Matthew Jaffee
fd50c15243 add fgprof endpoint by default
to aid in debugging performance issues
2021-11-05 14:02:58 -05:00
Souhaila Noor
590e5edc57 reverted to last working version 2021-11-05 13:57:37 -05:00
Souhaila Noor
7a5ba0e5d6 Revert "sanity check for scp"
This reverts commit 86407ebcab.
2021-11-05 13:52:26 -05:00
Souhaila Noor
cf365db944 changed ssh config 2021-11-05 13:47:47 -05:00
Souhaila Noor
98e5f6deaf ssh key 2021-11-05 13:45:11 -05:00
Souhaila Noor
4b2322160b sanity 2021-11-05 13:41:13 -05:00
seebs
f02c5c0354
Merge pull request #1748 from molecula/core932
FB-932: rework Import to reduce copying and network traffic
2021-11-05 13:38:12 -05:00
Souhaila Noor
fae8d2d116 ssh add command 2021-11-05 13:32:15 -05:00
Souhaila Noor
642d4fcb86 fixed variable name 2021-11-05 13:27:03 -05:00
Seebs
c7e4bc0fd0 refactoring experiment
I'm not actually sold on this, but I'm not entirely unsold on it. It seems like
it does reduce the amount of duplication a lot, but also it's sort of a mess.

In the process, noticed that it makes more sense to grab the whole cluster
rather than just the nodes for an arbitrary shard for the shard==^0 case,
because then if we have an API (but no Qcx), we can be reasonably confident
that we'll be able to pick the local node for loopback even if we aren't
using the API directly.

Have thought about whether we should create our own Qcx in cases like that
but I really don't like the idea of automatically creating a Qcx.
2021-11-05 13:06:38 -05:00
Seebs
d4b06d077e Import/ImportValue API rework and improvements
Underlying goal: Don't use the http client to send messages back to the
local host. Also, when sending data to other nodes, don't collate it
from an ImportRequest into a completely different format, then immediately
collate that back into an ImportRequest. This does require changing
the logic over in ctl/import to make it create an ImportRequest.

Also, add additional testing to make sure we're actually trying anything
at all with several combinations (such as submitting import requests
which don't match the configuration of index or field), and improve
test coverage for that.

This introduces the ability to tell an http/client InternalClient about
a specific API that it should use for local queries where applicable.
That's not implemented outside of the import stuff, but should probably
be applied eventually to other things that are trying to talk to many
nodes one of which may be the local node. That behavior is contingent
on passing in a Qcx, because it is implicitly tied to an existing
execution context, and it can't assume that it can create a new one,
because that could deadlock.
2021-11-05 13:06:38 -05:00
Seebs
60ac6a929f make address lookup failures okay on MacOS 2021-11-05 13:06:38 -05:00
Seebs
cff8da346f Improvements and benchmarks for ingest field operation sorting
Reusing the field operation sorting for other things caused me
to hit a bug, also made me curious about a performance issue and
whether it was possible to improve it. Answer: Not easy to improve,
anyway.
2021-11-05 13:06:38 -05:00
Souhaila Noor
4f964aeef4 updated ssh config 2021-11-05 12:53:39 -05:00
Souhaila Noor
86407ebcab sanity check for scp 2021-11-05 12:39:11 -05:00
Souhaila Noor
7c02e9c447 added scp 2021-11-05 12:37:15 -05:00
Ben Johnson
f2bc887d6f
Merge pull request #1751 from molecula/fb-904
[FB-904] Add deb/rpm packaging of FeatureBase binary
2021-11-05 10:59:48 -06:00
Souhaila Noor
d080164a49 echo aws config 2021-11-05 11:35:54 -05:00
Souhaila Noor
ab3679bef7 sanity check 2021-11-05 11:33:17 -05:00
Souhaila Noor
6e6d7a77db debug echo statements 2021-11-05 11:26:09 -05:00
tgruben
fd08815f51
Merge branch 'master' into fb-904 2021-11-05 11:23:01 -05:00
Souhaila Noor
a9ab597d28 added profile to start-instance 2021-11-05 11:20:21 -05:00
Souhaila Noor
021af2c5e5 changed ssh key name to match aws 2021-11-05 10:56:10 -05:00
Souhaila Noor
0cfd5f116d echo ssh file 2021-11-05 10:50:59 -05:00
Souhaila Noor
550a610b5b sanity checks 2021-11-05 10:46:54 -05:00
Souhaila Noor
f7497bfe8f pass ssh key to a file 2021-11-05 10:35:15 -05:00
Souhaila Noor
6643e3963e reverted back to variable for ssh 2021-11-05 10:28:36 -05:00
Souhaila Noor
04b64c0f30 add path for ssh keys 2021-11-05 09:54:37 -05:00
Samir Patel
6fa8f0a88e
Merge pull request #1749 from molecula/bug/index-bounds
[FB-965] featurebase import: check for data in requests
2021-11-05 09:49:50 -05:00
Souhaila Noor
bd440f4c23 updated permissions for ssh key 2021-11-05 09:43:19 -05:00
Souhaila Noor
d147c03e22 add ssh config in section with using ssh keys 2021-11-05 09:32:53 -05:00
Souhaila Noor
7271821ebe fixed path for ssh keys 2021-11-05 09:28:25 -05:00
Souhaila Noor
14a9662494 fixed yaml syntax error 2021-11-05 09:24:20 -05:00
Souhaila Noor
f9ff3973d0 fixed argument name 2021-11-05 09:17:19 -05:00
Samir Patel
139f4c9cbb
Merge branch 'master' into bug/index-bounds 2021-11-05 09:16:39 -05:00
Samir Patel
cb6a364890 more qcx defers 2021-11-05 09:12:28 -05:00
Souhaila Noor
383663673b configure ssh 2021-11-05 09:11:27 -05:00
Souhaila Noor
321d273dce fixed typo in instance name 2021-11-05 09:03:51 -05:00
Ben Johnson
94c6c4ee53
Merge branch 'master' into fb-904 2021-11-04 16:58:30 -06:00