Commit graph

16 commits

Author SHA1 Message Date
Todd Gruben
fffcb0b1e4 bug in arrowtable accounting 2023-04-07 15:07:24 -05:00
Todd Gruben
3e228263b1 arrow cache needed a lock 2023-04-07 11:55:57 -05:00
Todd Gruben
5c5260f0b6 singleton arrow allocator 2023-04-07 11:41:19 -05:00
Todd Gruben
617afc4c1d try again 2023-04-06 17:50:03 -05:00
Todd Gruben
0404aad84b try retain 2023-04-06 14:58:47 -05:00
Todd Gruben
af40c3564a global arrow allocator 2023-04-06 14:42:50 -05:00
Todd Gruben
7525f46295 refactor table cache to include allocator 2023-04-06 14:16:48 -05:00
Todd Gruben
d1332e0c67 cache loaded dataframe 2023-04-06 09:41:39 -05:00
tgruben
dba15669c6
fb-1915 Support large id's in NDJSON (#2290)
* uses json.Decoder to allow for large integer values in ndjson format in bulk import
2023-03-03 15:37:56 -06:00
tgruben
4e45f19ca0
parquet-info command to browse parquet files (#2230)
* parquet-info command to browse parquet files

* null support in parquet
2023-01-26 12:55:54 -06:00
tgruben
90e2808f52
bulk import support for parquet files (#2226)
* bulk import support form parquet files
2023-01-26 08:32:39 -06:00
Matthew Jaffee
6d4c1d9db1
Sup 294 pre sort command (#2209)
* first cut at pre-sort command that works on ndjson

* finish pre_sort command for CSV and JSON and add test

* try fixing golangci-lint

* remove some dumb lint checks

* more linter disabling

* take .golangci.yml from previous repo

* go fmt (facepalm)

* remove ioutil to fix lint
2023-01-23 12:26:38 -06:00
tgruben
adbad90fe1 string support for dataframe (#2405)
* String support for arrow

(cherry picked from commit 4cd3cb02a2)
2023-01-19 21:51:16 +00:00
tgruben
24372a9405 [FB-1822] Change dataframe disk format to Arrow from parquet (#2376)
* change default backend to arrow file format instead of parquet

(cherry picked from commit 6d96a1474e)
2023-01-10 23:22:58 +00:00
Fletcher Haynes
5c39a49285 Sync from private repo to commit 12d608c80d 2022-12-12 09:01:20 -08:00
tgruben
2fc7abd72e Dataframe (#2241)
* Dataframe

(cherry picked from commit 2f1beaf119)
2022-12-12 09:01:20 -08:00