* Add kafka support to CLI (fbsql)
This commit adds the ability to provide a `--kafka-config` command line
argument referncing a toml file to configure kafka.
* Move "Molecula Consumer" message to the logger; hide it in basic mode
* Fold decimal(scale) into kafka.source-type
* Build fbsql with cgo in docker for CI
* Re-organize the fbsql kafka config and setup.
Allow field config to use the table schema if no fields provided.
* Display timestamp fields with format RFC3339Nano
* remove kafkaRunner (no longer used)
* Fix cli/batch test (and make sure it's not excluded from CI)
The logic in our Makefile was exluding from tests any package with
`/batch` in the package name. This excluded `/cli/batch`, which is not
good.
This commit changes the exclusion logic to include the `/v3` portion of
the package name, so `/v3/batch`.
* Rename Basic() to SetBasic()
* Prevent file corruption when writing tar backup to stdout
FB-1794
Tar backups written to stdout were coming out corrupt. This turned
out to be due to log messages getting written to stdout and ending
up in the tar file. We now check to see if the tar file and the log
are both going to stdout, and if they are, send the logs to stderr
instead.
Testing did not have any kind of consistency or validity check. We
now compare a tar file sent to a file and a tar file sent to stdout
to make sure they're the same. This does not guarantee correctness
but does at least catch this form of corruption.
* trying different index name
Co-authored-by: tgruben <tgruben@gmail.com>
Co-authored-by: Todd Gruben <todd@molecula.com>
(cherry picked from commit a8996a149d)
stdout/stderr around
A lot of functions in the cmd and ctl packages were passing these
around and barely using them. Replaced them with a logger for most
functions. Some functions get an io.Writer instead so that their
tests can find the output they're looking for.
More cleanup on fb-1766: reworked the tests that were using io.Pipe
or os.Pipe to check their results so they now use a bytes.Buffer.
Unexported some variables that didn't need to be exported.
Fixed NewConfigCommand to use the provided stderr, not os.Stderr.
Added tests for rbf_dump, rbf_page, and keygen, since those weren't
being tested at all.
Added chksum_test, final cleanup.
(cherry picked from commit f627199acb)
Cobra automatically displays usage messages, and also a gratuitous
"Error: [...]" line in some cases, when any error at all occurs
running a command. To suppress the usage message, you have to set
cmd.SilenceUsage to true. But the code that would do this doesn't
have access to it. To address this, we introduce a category of
"usage error", implemented with stdlib error wrapping (%w) and
use errors.Is to check for it. There's also utility functions
to do this checking automatically, or indeed, to handle wrapping
of the ctl.SomethingCommand and handle running it with a suitable
context and everything.
In fact, several of the places we're checking for usage errors,
we can never actually report one, but we're checking consistently
so that if we want to report usage errors, we can.
For instance, server.Start and (dax)server.Start don't ever
return usage errors, right now, but we're checking their responses
anyway.
(cherry picked from commit c681642734)
