Switch Serverless from using BoltDB to Postgres as metadata store.
Previously, the controller stored all metadata to BoltDB. This implements SQLDB (currently Postgres flavored) as the backing store for metadata. This will allow us to have multiple instances of the controller running for HA, and to easily inspect and repair the contents of the metadata store.
Unfortunately, it was not straightforward to keep the BoltDB implementation working alongside the SQL one, so it will be removed in a later patch. Once that's done, the SQL implementation should allow for a number of simplifications of the schemar and balancer interfaces.
Database migration is built directly into the application by embedding the migration files and logic from the `soda` command line tool. When connecting to the RDBMS, the app will always attempt to create the necessary database and apply any outstanding migrations.
Integration tests truncate all tables upon start, but *not* at the end, so the state of the database can be inspected after integration tests.
Had to refactor some of the controller's background tasks to make sure they get properly shut down on controller exit.
* Implement Schemar.SetDatabaseOption(option, value string)
This replaces the temporary `SetDatabaseOptions()` method, which
replaced the entire DatabaseOptions struct, with `SetDatabaseOption`
which takes an option/value pair of strings to set.
* Add SetDatabaseOption to controller http handler and client
This commit also:
- renames some `writeLog` to `writelog`
- updates ApplyDirective to call resource.Unlock() on any resources
being removed from the local worker
* Add Database related methods to SchemaAPI interface
Currently all implementations of this interface are implemented with
"unimplemented" errors on those methods. Next will be to implement the
necessary methods.
* SQL: CREATE DATABASE and SHOW DATABASES
* SQL: DROP DATABASE
* SQL: Add UNITS option to CREATE DATABASE
* SQL: ALTER DATABASE
* User serverlessStorage.Remove[*]Resource instead of resource.Unlock()
* Add WITH keyword to CREATE/ALTER DATABASE
* fix some WITH logic
* linter fixes
* WITH on CREATE DATABASE is not required
* Remove MDS and replace it with Controller
This commit removes the MDS layer (and package) and shifts Controller
package into its place.
* add pprof/fgprof to serverless http router
---------
Co-authored-by: Matthew Jaffee <jaffee@pilosa.com>
2023-01-30 16:54:12 -06:00
Renamed from dax/mds/schemar/schemar.go (Browse further)
