empty string doesn't work because gitlab doesn't set the variable at
all. How do I know that "null" is correct? Because Fletcher told
me... apparently it's a ruby-ism
- For server side, used an instrumented binary with a test that wraps around the main entrypoint for featurebase
- Every time, the binary is called, a new coverage file is generated.
- For the client side, used the standard -coverprofile flag for go test to generate code coverage
- For backup test that's expected to fail, needed to call Run call in backup.go directly. The code coverage is not written to disk for an instrumented binary if there is an error.
We have pipelines that get to the gauntlet stage then get failed because
the ASG scales-in before the gauntlet stage finishes. (4/6 of the last
gauntlet failures were from this failure.)
There are a few ways to fix this, but my proposal is to turn on scale-in
protection to stop scaling in the instance running the gauntlet job
(scale in other instances instead), then turn off the scale-in
protection after the gauntlet test is run.
this should be a lot more reliable than trying to construct it based
on the project name as the exact construction can differ between
docker-compose versions.
There was also an issue with the backups succeeding when they should
fail in the test. There's an arcane maze of HTTP timeouts to navigate
here, but basically there are situations where the client will just
wait forever rather than erroring if the server is paused at the
right(wrong) time. I'm not convinced we've solved every possible case
of this, so we still may see the backup succeed even when it's
supposed to fail. The ultimate hammer is to add Client.Timeout, but
that's a very blunt instrument and I'm afraid it could cause a timeout
when really we just have a lot of data to download or something.
There may be a better way to say "only time out if you literally
haven't heard a peep from the server in this long", but I haven't been
able to figure it out yet.
I also fixed how the authclustertests are run as they weren't using
the PROJECT parameter correctly. Now they can run concurrently with
clustertests, and with other copies of authclustertests without having
conflicts.
- Add auth-token for featurebase import, backup and restore
- Add auth-token to http request
- Create a cluster tests with auth enabled
- Add test for import with auth enabled
gitlab CI runs as much as 5x slower sometimes during business hours,
resulting in tests failing due to 10-11 minute timeouts that would
succeed in under 2-3 minutes outside of business hours. to allow us
to do anything at all, let's just set that to half an hour, and 90
minutes for `go test -race`.
Concern: It's possible there's a timeout that's a gitlab CI configuration
thing involved too, because we see some go test timeout panics, but we
also see some weird messages about SIGQUIT at 11 minutes, which isn't
the go test timeout, so we may need to address that too.
Note that we're changing the Makefile, and also the config for the
gitlab CI passes, which don't use the Makefile. The Makefile changes
are just to be careful and avoid retriggering this later. We may
want to revert these if we get the other issues fixed.
