Commit graph

9 commits

Author SHA1 Message Date
Travis Turner
eb6c6e3105
Add kafka support to CLI (fbsql) (#2278)
* Add kafka support to CLI (fbsql)

This commit adds the ability to provide a `--kafka-config` command line
argument referncing a toml file to configure kafka.

* Move "Molecula Consumer" message to the logger; hide it in basic mode

* Fold decimal(scale) into kafka.source-type

* Build fbsql with cgo in docker for CI

* Re-organize the fbsql kafka config and setup.

Allow field config to use the table schema if no fields provided.

* Display timestamp fields with format RFC3339Nano

* remove kafkaRunner (no longer used)

* Fix cli/batch test (and make sure it's not excluded from CI)

The logic in our Makefile was exluding from tests any package with
`/batch` in the package name. This excluded `/cli/batch`, which is not
good.

This commit changes the exclusion logic to include the `/v3` portion of
the package name, so `/v3/batch`.

* Rename Basic() to SetBasic()
2023-03-07 08:18:22 -06:00
Pat Okeeffe
9386fc75b2
implement select from time quantum columns (fb-1654) (#2282)
* first time quantum queries working

* implement select from timequantum columns

* skip a dax test

* make linter happy

* addressed review feedback

* reverted over eager test elimination
2023-03-02 00:06:58 -06:00
Travis Turner
126be915a9
Support for setting individual DatabaseOptions (#2231)
* Implement Schemar.SetDatabaseOption(option, value string)

This replaces the temporary `SetDatabaseOptions()` method, which
replaced the entire DatabaseOptions struct, with `SetDatabaseOption`
which takes an option/value pair of strings to set.

* Add SetDatabaseOption to controller http handler and client

This commit also:
- renames some `writeLog` to `writelog`
- updates ApplyDirective to call resource.Unlock() on any resources
  being removed from the local worker

* Add Database related methods to SchemaAPI interface

Currently all implementations of this interface are implemented with
"unimplemented" errors on those methods. Next will be to implement the
necessary methods.

* SQL: CREATE DATABASE and SHOW DATABASES

* SQL: DROP DATABASE

* SQL: Add UNITS option to CREATE DATABASE

* SQL: ALTER DATABASE

* User serverlessStorage.Remove[*]Resource instead of resource.Unlock()

* Add WITH keyword to CREATE/ALTER DATABASE

* fix some WITH logic

* linter fixes

* WITH on CREATE DATABASE is not required
2023-02-06 08:50:28 -06:00
pokeeffe-molecula
5d025e399b implement CREATE/ALTER/DROP VIEW (fb-1592) (#2408)
* implement CREATE/ALTER/DROP VIEW

* fixed failing test

* another failing test

* fixed some broken serverless tests

(cherry picked from commit c620aae350)
2023-01-19 21:51:16 +00:00
Travis Turner
0127147d69 Thread Owner, UpdatedAt, UpdatedBy through SchemaAPI (#2351)
* Fix "qualifer" misspellings

* Remove `track_existence` and `shard_width` from SHOW TABLES output

* Thread Owner, UpdatedAt, UpdatedBy through SchemaAPI

I took the liberty of renaming "LastUpdatedUser" to "UpdateBy" to align
with "UpdatedAt".

(cherry picked from commit 63cfdb5078)
2023-01-10 23:19:19 +00:00
Fletcher Haynes
5c39a49285 Sync from private repo to commit 12d608c80d 2022-12-12 09:01:20 -08:00
Travis Turner
5c76ad5e70 Move batch.Importer interface to pilosa.Importer (#2347)
* Move batch.Importer interface to pilosa.Importer

In addition to moving the interface, it updates all the methods to use
dax.TableID (for example) intead of a string pilosa index name.

* Change unused onPremImporter methods to no-op.

onPremImporter is a wrapper around API which implements the Importer
interface. This is currently only used by sql3 running locally in standard
(i.e not "serverless") mode. Because sql3 always sets
`useShardTransactionalEndpoint = true`, There are several methods which this
implemtation of the Importer interface does not use, and therefore they
intentionally no-op.

(cherry picked from commit 12d608c80d)
2022-12-12 09:01:20 -08:00
Travis Turner
830b2ab4c8 Add Table.Description, Table.CreatedAt, Field.CreatedAt support to SchemaAPI (#2340)
* Thread Table.Description through SchemaAPI

* Thread Table.CreatedAt through SchemaAPI

* Thread Field.CreatedAt through SchemaAPI

(cherry picked from commit 734477aaee)
2022-12-12 09:01:20 -08:00
Travis Turner
4e3856348c Convert SchemaAPI interface to use dax.Table instead of pilosa.IndexInfo (#2336)
* WIP: Convert SchemaAPI to be DAX-centric

* Tables(), CreateField()

* CreateTable(), DeleteTable(), DeleteField()

* More cleanup

* Remove the old SchemaAPI

(cherry picked from commit a15783cb49)
2022-12-12 09:01:20 -08:00