Commit graph

28 commits

Author SHA1 Message Date
Matthew Jaffee
d4820d872f decode int base from schema in client code
not having this breaks ingest when ingesting int-like fields with
nonzero bases (using the shard transactional endpoint)
2022-05-29 09:03:27 -05:00
Matthew Jaffee
9fb68b416c add doc comments 2022-05-27 11:25:17 -05:00
Matthew Jaffee
772496b440 "all bitmap" multi-field, single-shard ingest
This adds a shard-based import endpoint which takes bitmap data for
all field types and imports data for the whole shard transactionally.

It uses the BitmapRewriter interface to try to intelligently allow for
setting and clearing bits simultaneously without multiple writes which
is especially helpful when ingesting into int-like fields, but also
allows clear-and-then-set behavior for set fields.
2022-05-27 11:25:17 -05:00
Samir Patel
43a61d87b2
return HTTP status code: 400 (Bad Request) when ingest values are (#2047)
out of range. Previously internal server error was returned.

This is to allow for ingest to continue while logging bad values
instead of stopping ingest as we do when there is a server error.
2022-05-17 14:26:10 -05:00
Travis Turner
7ccc845aac
Change Ttl to TTL (#2038)
* Change Ttl to TTL

Following go convention, acronyms should have a consistent case.
See
[Initialisms](https://github.com/golang/go/wiki/CodeReviewComments#initialisms)

This commit changes some public-facing methods, so any code importing
this package and using these methods will need to be updated.

* rewrite Ttl -> TTL

Co-authored-by: reesporte <reesedporter@gmail.com>
2022-04-27 11:20:39 -05:00
Samir Patel
17e8dbd318 client side retry ingestAPI requests on primary host
If non-primary host fails to process a request, retry on primary node.

conditions when we should not do this:
- no error
- we've aleady tried the primary
- we're making a status request to get the primary node...this
  could lead to lock contention if we allow it to happen as we are
  making an http request within an on going http request to discover
  the primary node.

This also deletes the RemoveHost method and the associated test b/c
it is not used anywhere anymore and updates the returned error type.
2022-03-10 10:22:13 -06:00
Hoang Pham
883e940d7f FB-1188 - Fixed units tests for TTL 2022-03-01 16:48:03 -06:00
Hoang Pham
e16171cb4b FB-1188 - Added TTL field option 2022-02-25 11:39:31 -06:00
Souhaila Noor
0e1cf5bbbd Enable authentication/authorization for featurebase tools
- Add auth-token for featurebase import, backup and restore
- Add auth-token to http request
- Create a cluster tests with auth enabled
- Add test for import with auth enabled
2022-01-26 17:30:26 -06:00
Ben Johnson
9ebf0e2119 Upgrade go.mod to featurebase/v3 2022-01-21 10:57:05 -07:00
Matthew Jaffee
a16fee5f88 set shardWidth properly in client
the shardwidth22 tests were broken client side, but we didn't realize
this because we weren't running the client side tests since moving the
client code into the main FB repo until recently (woops), and more
recently, we'd stopped running the shardwidth22 tests in the move to
Gitlab, so when we re-enabled them we finally noticed that they were
broken in the client.

All this change does is takes the shardWidth value from the core
featurebase package instead of using a hardcoded value in the client package.
2022-01-17 10:57:01 -06:00
Matthew Jaffee
b8da3bc7e6 checksum All() instead of Count(All()) to cover index keys 2021-12-10 11:52:39 -06:00
reesporte
48aef0c8a4 add copyright notice back in
```bash
for file in `cat diffys`; do
   printf '%s\n%s\n' "// Copyright 2021 Molecula Corp. All rights reserved." "$(cat $file)" >$file;
done
```
2021-12-10 11:01:04 -06:00
reesporte
4c53f86e82 removed license from each go file
i used this script, a little clunky but it got the job done

```bash
for file in `find . -type f -print | grep '\.go'`; do
    sed '1,/^\/\/ limitations under the License.$/d' $file > $file.tmp;
    result=`cat $file.tmp`
    if [[ result != "" ]]; then
        gofmt $file.tmp &> /dev/null;
        if [[ $? == 0 ]]; then
            mv $file.tmp $file && gofmt -w $file;
        else
            rm $file.tmp;
        fi
    else
        rm $file.tmp;
    fi
done
```
2021-12-10 09:17:17 -06:00
Matthew Jaffee
1a4180ca4a add new RecordBatch implementation which uses ingest API 2021-12-06 17:06:55 -06:00
Todd Gruben
2ddcbce8ad fix govet and gofmt errors in existing code 2021-10-29 13:14:27 -05:00
rachithrr
43b34c7d84 CORE-747: Build tooling to provide datagen-like functionality that uses the API
Featurebase
2021-09-02 10:18:50 -04:00
Mahesh Arumugam
858f889745 FeatureBase Renaming: changing go.mod module name for featurebase 2021-07-19 09:20:30 -07:00
Nia Weiss
c0ef297b11
standardize key translation on the find and create methods
This migrates existing code from the old TranslateKey(s) endpoints to the newer CreateKeys and FindKeys endpoints.
The CreateKeys and FindKeys endpoints were created previously as the TranslateKeys endpoint had no way to behave sanely when the looked-up key did not exist (the parallel-arrays representation did not have a good way to represent a missing key).
This change also removes the old TranslateKey(s) functions from the translation stores.
It leaves a wrapper emulating the TranslateKey(s) endpoints so that old idk still works for now.
2021-06-07 14:47:52 -04:00
Nia Weiss
f4ba34247f
remove attributes
Attributes are unmaintained and unused.
They have become more of a liability than a benefit.
This change eliminates them from the codebase.
The only user-visible change (assuming that attrs are not used) is that the attrs field will no longer appear in row JSON.
2021-05-14 10:28:08 -04:00
tgruben
813687d522
Merge branch 'timestamp-epoch' into timestamp-client 2021-04-14 11:37:40 -05:00
Todd Gruben
5737381d23 clientside timestamp support 2021-04-14 11:28:06 -05:00
Alan Bernstein
177c27dc31 Switch to new logger in client code (go-pilosa) 2021-04-12 22:25:13 -05:00
Kuba Podgórski
65cb2cdf73 Update internal endpoints 2021-03-25 18:00:56 +01:00
Kuba Podgórski
761190ec32 Fix internal endpoint 2021-03-25 16:44:05 +01:00
Kuba Podgórski
000f708c40 Address PR comments
+ some docs
2021-03-24 18:09:01 +01:00
Kuba Podgórski
1e5388dbe9 Move internals proto files into separate package (pb) 2021-03-22 20:28:00 +01:00
Kuba Podgórski
8131e807bf Add client package with go-pilosa implementation 2021-03-22 20:22:38 +01:00