Commit graph

42 commits

Author SHA1 Message Date
Seebs
9c216ef8a6 use testhook test cleanup
The testhook post-test hooks only work if you use a TestMain to
invoke them, otherwise the cleanups can be registered but never
actually get run. This deletes the etcd sockets, and temp
directories, that we created from our test runs. We also fix
the test creating a temp file directly to create it in a TempDir
(which gets cleaned up after the test), and fix the name of the
top-level tests displayed in TestMain.
2022-09-26 12:33:43 -05:00
Seebs
2052bb01d8 refactor testing to share clusters more often
When doing tests, we create a ton of one-off clusters. This
turns out to be expensive and slow. Fixing it is surprisingly hard.

Fundamentally: If we're sharing clusters, we need to use different
indexes for each test, to avoid clashes. This changes index names.
As a side-effect, this reorders many partition-based things, like
the order keys are returned in. Thus, to fix this, we change a lot
of tests to no longer depend on the *order* in which strings are
returned.

Having done that, we can also discard the ModHasher behavior, since
that only existed to allow us to reliably predict partitioning.

The basic design is as follows: Instead of a cluster being a
[]*Command, a "shareable" cluster is now a []*Command plus some
flags, and a "cluster" is a pointer to a possibly-shared cluster,
plus a link to the specific test using this specific cluster,
and correspondingly, its test name suitably coerced to be a valid
index name prefix.

The "test.Cluster" object now has methods to allow retrieving an
index name, and also implemnts fmt.Formatter to let you use,
e.g., `%i` with it in Sprintf to get "the index name, plus an i".
(This works for everything but %p and %T.)

This allows us to consistently rework all the many things that
use index names in a persistent way.

We also have `MustUnshared` and `MustRunUnsharedCluster` methods
which allow us to specify that a given test needs its own cluster
for some reason. For instance, the tests that want to run backups
need their own isolated cluster, and the tests that want to close
or reopen nodes need their own cluster because a reopened cluster
won't have working GRPC for some reason.

On "closing" a shared cluster (actually the test-specific wrapper
that reflects a given sharing), we delete any indexes starting with
that test's index name prefix. Otherwise, the huge pile of open
indexes prevents `go test -race` from working on MacOS, where we
run out of address space too quickly.

This is fairly enormous but most of the individual changes are
fairly trivial things like replacing the string "i" with "c.Idx()".

We also tweaked a test that failed for me a couple of times to
not depend on sort order.
2022-09-02 11:40:37 -05:00
Samir Patel
f08d3e2834
fix parens around where clause bug (#2121)
Bug: if a sql query had a where clause within parens, the entire
clause would be ignored; and instead of it translating to a pql
intersection, it would become an All().

This occured b/c the parser library mapped such an expresstion to
a sqlparser.ParenExpr, and we did not have this as a condition in
a type switch.

So instead of treating a ParenExpr as nothing, we now recurse into
it.
2022-06-16 22:21:24 -05:00
reesporte
9e17579a76 don't panic on nil field during inner join
when running a select statement with an inner join where the secondary field is non-existent, we get a panic. this commit fixes that.

see [fb-766](https://molecula.atlassian.net/browse/FB-766) for more information.
2022-06-14 10:01:16 -05:00
Seebs
727c2ed724 gracefully handle invalid indexes in joins
If a join is requested against an index that doesn't exist, we
should report that as an error rather than panicing.
2022-04-13 11:44:58 -05:00
Ben Johnson
1222bf22cd Use Distinct() call for SQL DISTINCT 2022-03-01 15:04:32 -07:00
Ben Johnson
7ebc28a734
Merge branch 'master' into fb-1226 2022-03-01 12:09:57 -07:00
Seebs
21a478a728 don't look up a field by name to find out its name
If a field doesn't exist, looking up that field produces a nil,
and querying the name of a nil field fails. Don't do that. Instead,
just use the name you're looking it up by.

We could in theory return an error here, but we already handle
nonexistent fields elsewhere and checking this when we already have
checks for it seems unnecessary, I think?

Also, we add a test for this. The test is over in server/grpc_test.go
because we have infrastructure there for testing the SQL server
functionality, and you can't actually write reasonable self-contained
tests for the SQL stuff because it has no way to create a working
server.
2022-03-01 11:49:42 -06:00
Ben Johnson
e69ad74532 Enable multi-field WHERE clause for GROUP BY SQL queries 2022-03-01 10:28:11 -07:00
Ben Johnson
0f70253cc0 Add SQL SELECT mapping test 2022-02-28 12:17:07 -07:00
Ben Johnson
9ebf0e2119 Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
reesporte
baf02748be filter http response and lockdown endpoints
- fixes required permissions on some http endpoints
- filters http endpoints:
    - /ui/usage
    - /schema
    - /schema/details
- filter GRPC show tables, fields
- allow admins to do anything
2022-01-14 16:05:54 -06:00
reesporte
48aef0c8a4 add copyright notice back in
```bash
for file in `cat diffys`; do
   printf '%s\n%s\n' "// Copyright 2021 Molecula Corp. All rights reserved." "$(cat $file)" >$file;
done
```
2021-12-10 11:01:04 -06:00
reesporte
4c53f86e82 removed license from each go file
i used this script, a little clunky but it got the job done

```bash
for file in `find . -type f -print | grep '\.go'`; do
    sed '1,/^\/\/ limitations under the License.$/d' $file > $file.tmp;
    result=`cat $file.tmp`
    if [[ result != "" ]]; then
        gofmt $file.tmp &> /dev/null;
        if [[ $? == 0 ]]; then
            mv $file.tmp $file && gofmt -w $file;
        else
            rm $file.tmp;
        fi
    else
        rm $file.tmp;
    fi
done
```
2021-12-10 09:17:17 -06:00
Todd Gruben
2e086b866a . 2021-11-03 09:37:52 -05:00
Todd Gruben
41ed706d83 add license 2021-11-03 09:02:47 -05:00
Todd Gruben
7fa71d9548 check selecthandler 2021-11-03 08:57:24 -05:00
Todd Gruben
2ddcbce8ad fix govet and gofmt errors in existing code 2021-10-29 13:14:27 -05:00
Todd Gruben
95dc4a1a50 basic looker connection tests pass
sql1 pass through works
2021-09-25 13:51:02 -05:00
Mahesh Arumugam
858f889745 FeatureBase Renaming: changing go.mod module name for featurebase 2021-07-19 09:20:30 -07:00
Nia Weiss
f4ba34247f
remove attributes
Attributes are unmaintained and unused.
They have become more of a liability than a benefit.
This change eliminates them from the codebase.
The only user-visible change (assuming that attrs are not used) is that the attrs field will no longer appear in row JSON.
2021-05-14 10:28:08 -04:00
Ben Johnson
cfc725e799 Add timestamp field type support 2021-04-06 10:50:10 -06:00
Travis
114f6a8751
add withViews argument to api.Schema() method 2021-02-08 10:42:55 -06:00
Antonio Navarro Perez
87ba73fa16 Stop writes on DEGRADED state
Signed-off-by: Antonio Navarro Perez <antnavper@gmail.com>
2021-02-04 17:30:14 +01:00
Alan Bernstein
eaf3ef153e Test SQL behavior 2021-01-20 15:58:55 -06:00
Alan Bernstein
a0d6c253d1 Pass SQL query string from mapper to tracker 2021-01-20 10:48:35 -06:00
Seebs
932e84b681 handling aggregate types: add to protobuf, etc
We want to distinguish different *kinds* of GroupCounts, so we're
making the GroupCounts parent object track its type so we can keep that
correct.

Adding this to protobuf, etc, then creates some weird behaviors
because sometimes we expect []GroupCount, and sometimes we expect
*GroupCounts. This implies changes to test cases. Also, the
changes to test cases imply that some test cases are probably now
wrong; for instance, they're expecting a "sum" column, equal to zero,
when no sum was requested.

We try to make the encoder handle a []*GroupCount gotten from another
node without panicing, and avoid breaking the semantics of the existing
messages, renumbering messages or components, etc.

Since a previous version, the `.Groups` member has been privatized,
and the `.Get()` convenience accessor has been renamed `.Groups()`
and is now used consistently in a way that should reduce the risk
of nil pointers causing crashes. Also, NewGroupCounts is used in
a couple more places.
2021-01-19 16:23:15 -06:00
Cody Soyland
4ebf6f6ff7 Customize serialization of []GroupCount based on aggregate type/presence 2021-01-19 12:10:56 -06:00
Nia Weiss
19e0f3601b
fix type names in PQL Extract and SQL Show 2020-09-15 12:53:21 -04:00
Nia Weiss
b676292d67
add named returns to clarify extractLimitOffset 2020-09-03 10:37:09 -04:00
Nia Weiss
daa784319c
remove SQL artificial limit 2020-09-03 09:14:11 -04:00
Travis
32b5826d1a
fix bug on left/right join mapping 2020-09-01 18:22:55 -05:00
Kuba Podgórski
ceb72fc3a9 support null results 2020-09-01 13:00:30 +02:00
Nia Weiss
d421558f58
fix SQL memory leak 2020-08-31 13:10:49 -04:00
Kuba Podgórski
25ead95967
Merge branch 'master' into grpc-errcode 2020-08-28 20:42:28 +02:00
Kuba Podgórski
2b1c9950f4 Add rich error types to gRPC interface 2020-08-28 17:24:49 +02:00
Travis
ae07aacd51
add sql mapper routes for count(*) on joins 2020-08-27 22:49:54 -05:00
Kuba Podgórski
2ca8ca0605 Pass name to newNotFoundError 2020-08-27 15:35:23 +02:00
Kuba Podgórski
397d37a129 Fix https://github.com/molecula/pilosa/issues/706 2020-08-24 15:34:09 +02:00
Kuba Podgórski
ceda4ab61b Add support for drop table 2020-08-24 13:24:18 +02:00
Kuba Podgórski
ad9e3338b5 Add support for SHOW queries 2020-08-22 01:25:35 +02:00
Kuba Podgórski
2b34976c22 porting sqlmapper from vdsm 2020-08-18 15:53:26 +02:00