Commit graph

316 commits

Author SHA1 Message Date
Souhaila Noor
0e1cf5bbbd Enable authentication/authorization for featurebase tools
- Add auth-token for featurebase import, backup and restore
- Add auth-token to http request
- Create a cluster tests with auth enabled
- Add test for import with auth enabled
2022-01-26 17:30:26 -06:00
reesporte
276088c386 fix panic on POST /transaction on non-primary node
- if we're a non-primary node, redirect to the primary
- if non-primary nodes can create transactions now, then the client should not receive an ErrNotPrimaryNode
- streamline metrics logic
2022-01-24 15:24:21 -06:00
reesporte
836df379ac add test coverage
for the following auth related packages:
* authn
* http
* server

fix minor bugs, do some cleaning up, etc in `authn/authenticate.go` and `http/handler.go`
2022-01-21 13:57:47 -06:00
Ben Johnson
9ebf0e2119 Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
reesporte
7644922406 adds logging to all network requests
addresses ticket FB-1109:
when auth is turned on, we log:
- source ip (if available)
- user-agent
- user id
- user name
- query string
- request endpoint

also adds some minor tweaks and comments  to chkAuthZ flow
2022-01-15 12:21:27 -06:00
reesporte
baf02748be filter http response and lockdown endpoints
- fixes required permissions on some http endpoints
- filters http endpoints:
    - /ui/usage
    - /schema
    - /schema/details
- filter GRPC show tables, fields
- allow admins to do anything
2022-01-14 16:05:54 -06:00
reesporte
cf2410fea6 addresses multiple authn/z tickets
* fb-998 - authn/z enabled in handlers (kitchen-sink ticket)
    - authorization is enabled through the use of a bearer token (using header "Authorization")
    - authorization may occur through the use of an "Authorization" header or "molecula-chip" cookie
    - ui is updated for changes to handler
* fb-1131 - protect grpc endpoints
    - GRPC endpoints now check authorization if auth is enabled
* fb-1129 - inter-node communication
    - the following endpoints use the secretKey for authentication:
        - /internal/cluster/message: POST
        - /internal/translate/data: GET, POST

* added test to api_test.go (TestAuth_MultiNode) testing various auth/permissions stuff on a multi-node cluster

not included:
    - fb-1130 - filter response of endpoints
    - fb-1109 - improved audit logging

@jaffee [are you not entertained](https://www.youtube.com/watch?v=mutgotxrcqg)

Co-authored-by: souhailanoor <90720110+souhailanoor@users.noreply.github.com>
Co-authored-by: tgruben <tgruben@gmail.com>
Co-authored-by: 54mir <48686912+54mir@users.noreply.github.com>
Co-authored-by: kcrodgers24 <49999391+kcrodgers24@users.noreply.github.com>
2022-01-14 12:31:32 -06:00
Samir Patel
41bde6ccba don't write content to no content 2022-01-06 10:35:11 -06:00
Samir Patel
6e9efd0e09 Merge branch 'protect-endpoints' of github.com:molecula/featurebase into protect-endpoints 2022-01-05 17:30:18 -06:00
Samir Patel
3fe381ff22 address feeback 2022-01-05 17:29:59 -06:00
reesporte
8d6490329b Merge branch 'master' into protect-endpoints 2022-01-04 16:18:18 -06:00
Samir Patel
414dff1d22 revisions 2022-01-03 22:56:21 -06:00
Samir Patel
1b10f26258 fix permission stuff for write queries 2022-01-03 17:41:09 -06:00
Samir Patel
7834db2347 change write call detection 2022-01-03 16:24:14 -06:00
Samir Patel
d18b739402 add test cases 2022-01-03 11:49:38 -06:00
Samir Patel
c0fe253ce2 Merge branch 'protect-endpoints' of github.com:molecula/featurebase into protect-endpoints 2021-12-29 14:41:49 -05:00
Samir Patel
cf86be16c1 add authN only middleware for /internal 2021-12-29 14:41:31 -05:00
reesporte
8e697c0d8c Merge branch 'protect-endpoints' of github.com:molecula/featurebase into protect-endpoints 2021-12-29 13:38:55 -06:00
reesporte
17679eb924 create a Permissions type
makes it nice to say p.Satisfies(otherPerm)
2021-12-29 13:38:11 -06:00
Samir Patel
7750900310 more logging 2021-12-29 13:18:42 -05:00
reesporte
42f3557c55 fix merge conflicts 2021-12-29 09:05:53 -06:00
Samir Patel
d95d4dac9d pass group membership thru context 2021-12-28 17:36:52 -05:00
Matthew Jaffee
1a8c10d5f3 fix backup fail test so it actually fails
A few things were going wrong here.

First, we take a "RetryPeriod" option on backup and restore which is
meant to be roughly the total amount of time we spend retrying any
given request before failing. However we were incorrectly passing that
as the RetryMaxWait which is the maximum amount of time to sleep
between any two attempts. We now do some fuzzy math to figure out
approximately how many attempts we should make given a minimum sleep
of 100ms and the fact that we double the sleep time every attempt.

Second, during the backup test, if a host was totally stopped when we
started the request, it would fail immediately and then retry, but if
the host was stopped during the request (after DNS had resolved), then
the request would wait for the DialTimeout which we default to 30s, so
turning off the cluster for 5 seconds and turning it back on resulted
in the backup completing rather than failing. Because of this, we
change the commandClient to have a default dial timeout of 1 second.

I was tempted to change the global default to 1s which I think would
be fine, but didn't want to break anything too badly.
2021-12-28 13:31:42 -06:00
Samir Patel
e5fa99a531 apply mw to handlers 2021-12-28 10:08:44 -05:00
Samir Patel
6d590581d4 apply mw to handlers 2021-12-28 10:06:40 -05:00
Samir Patel
7544e7d1cb extend mw 2021-12-28 09:24:46 -05:00
Samir Patel
b9961870c1 implement as mw 2021-12-27 18:22:53 -05:00
Samir Patel
7a6595d628 authorize few endpoints e.g. query 2021-12-27 16:43:47 -05:00
Ben Johnson
9367a62609 Add /debug/rbf endpoint for debugging 2021-12-27 09:34:43 -07:00
Samir Patel
49e9faa03b stub out checker 2021-12-22 16:49:43 -06:00
Samir Patel
684c408b93 Merge branch '54mir/protect-endpoints' into queryLoggerSetup 2021-12-22 15:12:31 -06:00
Samir Patel
52d941d127 more tests 2021-12-22 11:31:37 -06:00
Matthew Jaffee
cde3f6b5ea add profiling to backup/restore 2021-12-21 16:24:21 -06:00
rachithrr
5650a24c9b query logger is set up. 2021-12-21 16:52:22 -05:00
Samir Patel
3b374a62bf Merge branch 'master' into 54mir/authentication 2021-12-20 16:42:11 -06:00
Samir Patel
e7f4eb1e36 response codes 2021-12-20 12:28:17 -06:00
Samir Patel
67d438aab5 clean up 2021-12-20 01:29:11 -06:00
Samir Patel
0d52a952e0 resolve some comments 2021-12-19 23:33:44 -06:00
Samir Patel
3b58e887ed add group lenth check 2021-12-17 11:34:48 -06:00
Samir Patel
b37f13e5c5 rename 2021-12-16 20:33:23 -06:00
Samir Patel
52625c70ab check auth enabled before handling auth requests 2021-12-15 15:42:56 -06:00
Samir Patel
0cb27bfdd3 Merge branch '54mir/authentication' of github.com:molecula/featurebase into 54mir/authentication 2021-12-15 14:54:41 -06:00
Samir Patel
213572bb78 add logout and userinfo endpoints 2021-12-15 14:51:45 -06:00
Samir Patel
bd81427dd5 load auth object into handler 2021-12-13 23:10:04 -06:00
Hoang Pham
583a0293ce added Login page for testing with BE endpoint 2021-12-13 14:04:41 -06:00
Samir Patel
b5c87e0f18 add auth endpoints 2021-12-11 00:11:30 -06:00
Matthew Jaffee
7a8f0135b3 redirect GetTranslateData if node doesn't own partition 2021-12-10 11:52:39 -06:00
reesporte
48aef0c8a4 add copyright notice back in
```bash
for file in `cat diffys`; do
   printf '%s\n%s\n' "// Copyright 2021 Molecula Corp. All rights reserved." "$(cat $file)" >$file;
done
```
2021-12-10 11:01:04 -06:00
reesporte
4c53f86e82 removed license from each go file
i used this script, a little clunky but it got the job done

```bash
for file in `find . -type f -print | grep '\.go'`; do
    sed '1,/^\/\/ limitations under the License.$/d' $file > $file.tmp;
    result=`cat $file.tmp`
    if [[ result != "" ]]; then
        gofmt $file.tmp &> /dev/null;
        if [[ $? == 0 ]]; then
            mv $file.tmp $file && gofmt -w $file;
        else
            rm $file.tmp;
        fi
    else
        rm $file.tmp;
    fi
done
```
2021-12-10 09:17:17 -06:00
Seebs
8b20406f45 experiment: handle errors more gracefully, but don't stream CreateKeys
We were trying to write an error to a ResponseWriter After attempting to
write to it, and this produces messages about superfluous WriteHeaders,
which is correct. This patch changes things so that we report messages
more clearly and verbosely if we hit them before writing, and if we try
to write and fail, we log the message because that's all we can do.

This does change semantics slightly, in that now we're marshalling
separately from trying to write the marshalled data. I think this is
probably a reasonable call because it lets us get diagnostics about a
hypothetical encoding problem, but in practice I don't think there
should be any encoding problems. So my guess is the actual error will
occur in that last line, and be logged to the server console instead
of failing to write over HTTP.

Also note that this changes some of the messages to include the
underlying error they're complaining about.

We also merge the create/find and index/field cases because only a
couple of lines of code changed between four largeish functions,
and we test some of the failure cases.

We don't have test coverage on the "field isn't provided" type things
because the mux won't actually route things there without them, so
far as I know.
2021-12-01 11:06:54 -06:00