Commit graph

14 commits

Author SHA1 Message Date
reesporte
eab6174388 make pql.Decimal.Value a private big.Int field
This ensures that we can't overflow when adding `pql.Decimal`s together. The
only place we can possibly overflow is when converting pql.Decimal to an Int64,
but that is a risk we have to take. Also, the only place we do this is in our
ToRowser. We could maybe change that to strings, so the presentation of data
doesn't indicate an overflow, but that is a later decision to make. It will
also involve fixing the generate-proto-grpc make command, because that's broken
rn.
2022-06-06 16:44:01 -05:00
Matthew Jaffee
772496b440 "all bitmap" multi-field, single-shard ingest
This adds a shard-based import endpoint which takes bitmap data for
all field types and imports data for the whole shard transactionally.

It uses the BitmapRewriter interface to try to intelligently allow for
setting and clearing bits simultaneously without multiple writes which
is especially helpful when ingesting into int-like fields, but also
allows clear-and-then-set behavior for set fields.
2022-05-27 11:25:17 -05:00
Matthew Jaffee
0d50bd2890 implement ability to update TTL on time fields 2022-05-04 11:04:38 -05:00
Travis Turner
7ccc845aac
Change Ttl to TTL (#2038)
* Change Ttl to TTL

Following go convention, acronyms should have a consistent case.
See
[Initialisms](https://github.com/golang/go/wiki/CodeReviewComments#initialisms)

This commit changes some public-facing methods, so any code importing
this package and using these methods will need to be updated.

* rewrite Ttl -> TTL

Co-authored-by: reesporte <reesedporter@gmail.com>
2022-04-27 11:20:39 -05:00
Hoang Pham
e16171cb4b FB-1188 - Added TTL field option 2022-02-25 11:39:31 -06:00
reesporte
48aef0c8a4 add copyright notice back in
```bash
for file in `cat diffys`; do
   printf '%s\n%s\n' "// Copyright 2021 Molecula Corp. All rights reserved." "$(cat $file)" >$file;
done
```
2021-12-10 11:01:04 -06:00
reesporte
4c53f86e82 removed license from each go file
i used this script, a little clunky but it got the job done

```bash
for file in `find . -type f -print | grep '\.go'`; do
    sed '1,/^\/\/ limitations under the License.$/d' $file > $file.tmp;
    result=`cat $file.tmp`
    if [[ result != "" ]]; then
        gofmt $file.tmp &> /dev/null;
        if [[ $? == 0 ]]; then
            mv $file.tmp $file && gofmt -w $file;
        else
            rm $file.tmp;
        fi
    else
        rm $file.tmp;
    fi
done
```
2021-12-10 09:17:17 -06:00
reesporte
af3c5809e2 add support for multi-node queries 2021-12-06 14:57:05 -06:00
reesporte
fa635874d4 encode/decode timestamp val appropriately 2021-11-22 10:08:18 -06:00
Seebs
610c4ed6cb introduce protobuf types for ingest ops
We add a new protobuf type. Also, protoc changed slightly and remade
some tests, in a way which should have no effects but makes the code
*very* slightly cleaner.

This introduces the first testing code in encoding/proto (whoops)
so that scaffolding is a first draft; if you're looking at this code
and the design is a problem go ahead and fix it.

The purpose of this is to verify that we're actually covering all
the branches in the ingest.ShardedRequest and pb.ShardedIngestRequest
message conversions. (Except the top-level one for a nil request,
which isn't checked by this.)

The coverage report doesn't actually include coverage for the ingest
code, though, so we haven't actually properly tested Compare.
Baby steps!
2021-09-27 12:05:57 -05:00
Ben Johnson
d16978f5dc Add max memory limit to Extract() to prevent OOM
This commit changes the Extract() query to return an error if the
result set gets too large in order to prevent out-of-memory (OOM)
panics.
2021-08-02 08:20:12 -06:00
Nia Weiss
f4ba34247f
remove attributes
Attributes are unmaintained and unused.
They have become more of a liability than a benefit.
This change eliminates them from the codebase.
The only user-visible change (assuming that attrs are not used) is that the attrs field will no longer appear in row JSON.
2021-05-14 10:28:08 -04:00
Ben Johnson
d70eb737cb Fix timestamp field issues 2021-04-09 08:26:12 -06:00
Kuba Podgórski
1e5388dbe9 Move internals proto files into separate package (pb) 2021-03-22 20:28:00 +01:00