We want to retry our terraform setup if it fails, so let's check whether
it worked and possibly retry.
This loop is awful because I'm trying to both check the exit status
and the reported IPs. Once I know whether the exit status predicts the
reported IPs that should go away.
This is a bit complicated and entangled, sorry.
First, we squash the auth-based smoke tests into the regular smoke
tests; we just run all the tests with auth on and that way we don't
need to spin up an entire separate cluster of machines just to run
a single query against them.
We improve the error detection, and standardize the jq-to-get-config
code. The purpose of this is to try to make sure that, if we actually
hit a failure and get "null" for a host name, we report *that*
as an error, rather than running ahead and producing 20+ separate
reports that ssh failed because it couldn't find a host named null.
* unifying idk and featurebase: first pass
* resolved conflict with master for gitignore & dockerignore
* deleted binaries that were accidentally pushed to git
* combined gitlab jobs for idk & featurebase
* run go fmt for idk
* updated ssh env variable, and made docker password variable in gitlab env variables
* fixed typo assigning variable name
* trying to fix docker login error
* trying a different solution for docker password
* pass registry
* fixed docker login
* updated paths for idk
* exclude idk tests from featurebase test run
* fix vendor error
* update certificates
* grpc needs to be in version 1.38
genproto, which is imported by big query updates the grpc version to 1.47.0
grpc 1.47.0 causes etcd to deadlock when calling etcd.Close()
the fix is to have a replace in go.mod to specify a specific grpc version
* run go mod tidy
* go mod
* run go mod tidy
* exclude bigquery since it is causing issues and undo grpc replace in go.mod
* fix grpc version
* fix formatting error
* update formatting
* attempt to fix formatting
* update path for code coverage
* update to use current branch binaries, not master
* fix for building idk - path updates
* udpate path for binaries
* update job dependecies
* update docker idk tests to use the current branch registry
* update stages for jobs
* updated job dependencies
* not allow idk s3 dump to fail since it is a dependency for integration tests
* update dependecy for idk tests
* update paths for idk build and code coverage
* download featurebase binary from s3
* pass branch name to all setup scripts
* change to current branch instead of master
* updated sonarcloud
* sonarcloud fix and branch name fix
* trying to speed up pipeline run time
* update stage
* branch name fix + sonar cloud
* sonarcloud
* Use IP whitelisting for ingest
For ingest, use configured IPs to authenticate the requests.
Auth-token will no longer be used for requests from ingest consumers.
If IP in request is in configured IPs, authenticate and authorize as an admin.
If IP in request is not in configured IPs, proceed with the standard authentication/authorization using ADD.
* need to remove port from client IP
* addressed review comments
- terraform scripts to set up cluster
- cloud-formation scripts to set up cluster
- set up ingest node with kafka server and datagen
- set up second ingest node with molecula-consumer-kafka-static
- set up datadog in all nodes (ingest + featurebase)
- set up script to execute different queries
- only run delete test on schedule
this stems from https://molecula.atlassian.net/browse/SUP-194 where the string
"standard" was being passed to viewTimePart, which output "standard" as the result.
this is not a valid time string and was causing confusing errors. now it simply
doesn't do that
this commit also adds regression testing framework and a regression test for fb-1287
