- check that serverlessStorage is not nil before closing it
- check that we don't already hold a lock on a serverless storage
Manager before trying to load it. This fixed at least one test failure.
(cherry picked from commit 87d1c31607)
* Prevent file corruption when writing tar backup to stdout
FB-1794
Tar backups written to stdout were coming out corrupt. This turned
out to be due to log messages getting written to stdout and ending
up in the tar file. We now check to see if the tar file and the log
are both going to stdout, and if they are, send the logs to stderr
instead.
Testing did not have any kind of consistency or validity check. We
now compare a tar file sent to a file and a tar file sent to stdout
to make sure they're the same. This does not guarantee correctness
but does at least catch this form of corruption.
* trying different index name
Co-authored-by: tgruben <tgruben@gmail.com>
Co-authored-by: Todd Gruben <todd@molecula.com>
(cherry picked from commit a8996a149d)
* Make interfaces more specific than "MDS"
- Introduce `dax.Schemar` interface
- Introduce `dax.Noder` interface
- The rest is generally to standardize on the new interfaces.
- Remove `pilosa.SchemaInfoAPI` interface
- Move `TranslateNode` and `ComputeNode` types from controller to dax package
- Remove `queryer.FeatureBaseImporter`
- Remove `queryer.MDS` interface
- Remove `queryer.Importer` interface
- Identify types using an "MDS" interface and split into Noder/Schemar as necessary
- Changed `Queryer.orchestrator` to a `map[qual]*qualifiedOrchestrator` because we can't share an orchestrator across quals
* Convert orchestrator to use TableKeyer
(cherry picked from commit 14f1930004)
* Fix "qualifer" misspellings
* Remove `track_existence` and `shard_width` from SHOW TABLES output
* Thread Owner, UpdatedAt, UpdatedBy through SchemaAPI
I took the liberty of renaming "LastUpdatedUser" to "UpdateBy" to align
with "UpdatedAt".
(cherry picked from commit 63cfdb5078)
This commit introduces an interface called `TableKeyer` which anything that means to represent a "table"
can implement. Examples are `dax.QualifiedTable`, `dax.Table`, and `string` (for legacy pilosa calls
where Execute simply took `index string`).
In the case of `orchestrator.Execute()` and `qualifiedOrchestrator.Execute()`, we are intentionally strict
about which type of `TableKeyer` the respective method accepts. If we find, in the future, this is too
restrictive, we can loosen that; but for now it helps us understand what is expected.
(cherry picked from commit a61d1a9571)
* Move batch.Importer interface to pilosa.Importer
In addition to moving the interface, it updates all the methods to use
dax.TableID (for example) intead of a string pilosa index name.
* Change unused onPremImporter methods to no-op.
onPremImporter is a wrapper around API which implements the Importer
interface. This is currently only used by sql3 running locally in standard
(i.e not "serverless") mode. Because sql3 always sets
`useShardTransactionalEndpoint = true`, There are several methods which this
implemtation of the Importer interface does not use, and therefore they
intentionally no-op.
(cherry picked from commit 12d608c80d)
* Thread Table.Description through SchemaAPI
* Thread Table.CreatedAt through SchemaAPI
* Thread Field.CreatedAt through SchemaAPI
(cherry picked from commit 734477aaee)
* This adds in support to the lattice UI application to use the SQL3
endpoint. If the `/sql` endpoint returns 404, it will use the SQL1
endpoint. If the `/sql` endpoint is available, it will send SQL queries
to that. It does not try the SQL1 endpoint if the SQL3 endpoint returns
an error processing the query. That is, it is all SQL3 or SQL1.
- Below are the specific changes:
- Adds a file that contains functions for interacting with http services as opposed to just grpc/event-based services. As of this commit, it is only the SQL3 endpoint.
- This adds a variable to track if we are using the SQL3 endpoint or not
- This adds a function to handle the response from the SQL3 endpoint
- Adds a function to eventServices to query the sql3 HTTP endpoint
- Fixed a missing semicolon in grpcServices
Co-authored-by: Fletcher Haynes <fletcher.haynes@generalassemb.ly>
(cherry picked from commit 7ab453e289)
* WIP: Convert SchemaAPI to be DAX-centric
* Tables(), CreateField()
* CreateTable(), DeleteTable(), DeleteField()
* More cleanup
* Remove the old SchemaAPI
(cherry picked from commit a15783cb49)
* Move batch.Importer interface to pilosa.Importer
In addition to moving the interface, it updates all the methods to use
dax.TableID (for example) intead of a string pilosa index name.
* Change unused onPremImporter methods to no-op.
onPremImporter is a wrapper around API which implements the Importer
interface. This is currently only used by sql3 running locally in standard
(i.e not "serverless") mode. Because sql3 always sets
`useShardTransactionalEndpoint = true`, There are several methods which this
implemtation of the Importer interface does not use, and therefore they
intentionally no-op.
(cherry picked from commit 12d608c80d)
* Thread Table.Description through SchemaAPI
* Thread Table.CreatedAt through SchemaAPI
* Thread Field.CreatedAt through SchemaAPI
(cherry picked from commit 734477aaee)
* This adds in support to the lattice UI application to use the SQL3
endpoint. If the `/sql` endpoint returns 404, it will use the SQL1
endpoint. If the `/sql` endpoint is available, it will send SQL queries
to that. It does not try the SQL1 endpoint if the SQL3 endpoint returns
an error processing the query. That is, it is all SQL3 or SQL1.
- Below are the specific changes:
- Adds a file that contains functions for interacting with http services as opposed to just grpc/event-based services. As of this commit, it is only the SQL3 endpoint.
- This adds a variable to track if we are using the SQL3 endpoint or not
- This adds a function to handle the response from the SQL3 endpoint
- Adds a function to eventServices to query the sql3 HTTP endpoint
- Fixed a missing semicolon in grpcServices
Co-authored-by: Fletcher Haynes <fletcher.haynes@generalassemb.ly>
(cherry picked from commit 7ab453e289)
* WIP: Convert SchemaAPI to be DAX-centric
* Tables(), CreateField()
* CreateTable(), DeleteTable(), DeleteField()
* More cleanup
* Remove the old SchemaAPI
(cherry picked from commit a15783cb49)
* FB-1739: Add ability to add a description to a table on creation
- Added CommentOption to handle text after COMMENT option.
- added description field in the createtable plan.
- The description is stored in the existing index metadata.
(cherry picked from commit ad350c2d49)
* Clean up dax service interfaces
Rename some of the `computer` interfaces and organize them in the
appropriate files.
Remove `dax/computer/alpha` package
* Remove ComputeAPI (it was replaced by batch.Importer)
* add nss-tools dependecy to smoke test
(cherry picked from commit 969bf055b2)
* moved the debug code to the right spot
* enforce int min/max constraints on inserts
* add a check for decimal min and max
* fixed borked tests
* fix the decimal to int conversion in constraint check
Co-authored-by: Travis Turner <travis@molecula.com>
(cherry picked from commit e392ce3460)
