Commit graph

38 commits

Author SHA1 Message Date
Matthew Jaffee
a16fee5f88 set shardWidth properly in client
the shardwidth22 tests were broken client side, but we didn't realize
this because we weren't running the client side tests since moving the
client code into the main FB repo until recently (woops), and more
recently, we'd stopped running the shardwidth22 tests in the move to
Gitlab, so when we re-enabled them we finally noticed that they were
broken in the client.

All this change does is takes the shardWidth value from the core
featurebase package instead of using a hardcoded value in the client package.
2022-01-17 10:57:01 -06:00
Matthew Jaffee
dad244e0d3 skip sometimes-failing test of experimental code
this is killing us in CI for no good reason
2022-01-13 13:16:57 -06:00
Matthew Jaffee
48b4169cb5 refactor client batch tests to reduce duplication
also use a single cluster with each test creating a different index
rather than each test creating a whole new cluster.

runtime went from 38s to 30s in my informal tests
2022-01-11 10:17:43 -06:00
Matthew Jaffee
db87a3c4f7 fix vet shadow issue 2022-01-11 10:15:58 -06:00
Matthew Jaffee
7fbd371038 get some of the client tests to actually *run*
discovered that client tests weren't running due to integration build
tag. Fixed the file I needed to get through SonarCloud and documented
rest of what needs to be done in FB-1152 https://molecula.atlassian.net/browse/FB-1152
2022-01-11 10:15:58 -06:00
Matthew Jaffee
131f891f75 fix up error messages in client batch test 2022-01-11 10:15:58 -06:00
Matthew Jaffee
55a385ed2d add sorting for ints/mutex in batch importer
fixes pathological case where imports with randomly ordered IDs which
spanned multiple shards and included ints or mutex fields could be
incredibly slow due to making 1000s of requests.
2022-01-11 10:15:58 -06:00
Matthew Jaffee
b8da3bc7e6 checksum All() instead of Count(All()) to cover index keys 2021-12-10 11:52:39 -06:00
reesporte
48aef0c8a4 add copyright notice back in
```bash
for file in `cat diffys`; do
   printf '%s\n%s\n' "// Copyright 2021 Molecula Corp. All rights reserved." "$(cat $file)" >$file;
done
```
2021-12-10 11:01:04 -06:00
reesporte
4c53f86e82 removed license from each go file
i used this script, a little clunky but it got the job done

```bash
for file in `find . -type f -print | grep '\.go'`; do
    sed '1,/^\/\/ limitations under the License.$/d' $file > $file.tmp;
    result=`cat $file.tmp`
    if [[ result != "" ]]; then
        gofmt $file.tmp &> /dev/null;
        if [[ $? == 0 ]]; then
            mv $file.tmp $file && gofmt -w $file;
        else
            rm $file.tmp;
        fi
    else
        rm $file.tmp;
    fi
done
```
2021-12-10 09:17:17 -06:00
Matthew Jaffee
77897f3ef0 try to get some more test coverage on error cases
(without creating too much duplication!)
2021-12-06 20:19:42 -06:00
Matthew Jaffee
1a4180ca4a add new RecordBatch implementation which uses ingest API 2021-12-06 17:06:55 -06:00
reesporte
666baffb7d Merge branch 'master' into staticcheck-issues 2021-12-03 09:36:13 -06:00
reesporte
63c5c11108 fix some staticcheck issues 2021-12-03 09:31:45 -06:00
Todd Gruben
1df90af26e free bitmap pages on deallocate 2021-12-02 15:57:52 -06:00
Todd Gruben
2ddcbce8ad fix govet and gofmt errors in existing code 2021-10-29 13:14:27 -05:00
kcrodgers24
1f257aab9c adds more detail to CSV ingest error message 2021-10-08 08:27:49 -07:00
rachithrr
43b34c7d84 CORE-747: Build tooling to provide datagen-like functionality that uses the API
Featurebase
2021-09-02 10:18:50 -04:00
Matthew Jaffee
0d7f1c49c4 need to create rowCache lock at the top... otherwise it does nothing 2021-07-20 12:15:35 -05:00
Matthew Jaffee
d2e8c5f773 fix other batch client rowCache race 2021-07-20 10:36:17 -05:00
Matthew Jaffee
11686cac97 add locking around rowCache to avoid data race 2021-07-20 09:00:49 -05:00
Mahesh Arumugam
858f889745 FeatureBase Renaming: changing go.mod module name for featurebase 2021-07-19 09:20:30 -07:00
Nia Weiss
539d1ffe6b
fix batch import of time fields with key translation
Time fields were not listed as a type which could accept key translation, causing the translation code to fail.
This also adds tests for keyed time and mutex fields.
2021-06-10 09:41:59 -04:00
Nia Weiss
c2f58f7af0
fix column key translation test
It previously checked hardcoded values.
This does not work due to shard width differences.
2021-06-07 14:52:04 -04:00
Nia Weiss
39337fa41b
fix shadowing of require in client internal tests
The `require` tool was being shadowed with the outer test's T, causing the testing package to explode sometimes.
2021-06-07 14:52:04 -04:00
Nia Weiss
c0ef297b11
standardize key translation on the find and create methods
This migrates existing code from the old TranslateKey(s) endpoints to the newer CreateKeys and FindKeys endpoints.
The CreateKeys and FindKeys endpoints were created previously as the TranslateKeys endpoint had no way to behave sanely when the looked-up key did not exist (the parallel-arrays representation did not have a good way to represent a missing key).
This change also removes the old TranslateKey(s) functions from the translation stores.
It leaves a wrapper emulating the TranslateKey(s) endpoints so that old idk still works for now.
2021-06-07 14:47:52 -04:00
Nia Weiss
f4ba34247f
remove attributes
Attributes are unmaintained and unused.
They have become more of a liability than a benefit.
This change eliminates them from the codebase.
The only user-visible change (assuming that attrs are not used) is that the attrs field will no longer appear in row JSON.
2021-05-14 10:28:08 -04:00
tgruben
813687d522
Merge branch 'timestamp-epoch' into timestamp-client 2021-04-14 11:37:40 -05:00
Todd Gruben
5737381d23 clientside timestamp support 2021-04-14 11:28:06 -05:00
Alan Bernstein
177c27dc31 Switch to new logger in client code (go-pilosa) 2021-04-12 22:25:13 -05:00
Samir
83932041a5 Replicate timefield functionality in pilosaclient 2021-04-08 13:21:49 -05:00
Kuba Podgórski
c23bc901e8 Integration tests client against cluster 2021-03-29 18:31:45 +02:00
Kuba Podgórski
9a02004a4f Move vprint to separate pakage 2021-03-29 14:29:19 +02:00
Kuba Podgórski
65cb2cdf73 Update internal endpoints 2021-03-25 18:00:56 +01:00
Kuba Podgórski
761190ec32 Fix internal endpoint 2021-03-25 16:44:05 +01:00
Kuba Podgórski
000f708c40 Address PR comments
+ some docs
2021-03-24 18:09:01 +01:00
Kuba Podgórski
1e5388dbe9 Move internals proto files into separate package (pb) 2021-03-22 20:28:00 +01:00
Kuba Podgórski
8131e807bf Add client package with go-pilosa implementation 2021-03-22 20:22:38 +01:00