mirror of
https://github.com/featurebasedb/featurebase.git
synced 2026-09-07 17:15:56 +00:00
cf5a8c8382
The `build lattice` job is a source of frustration because it is a dependency for building additional jobs in CI, has a wildly varying runtime--anywhere from 2-7 minutes to finish--and doing a yarn install && yarn build is the most CPU/memory hungry part of our CI pipeline. To address this, I'm hashing the lattice directory, and using the hash to look for an artifact in S3 which can be downloaded instead of building the web ui from scratch each commit. This cache happy path (cached file exists in S3) cuts the time of this job down to somewhere below 2 minutes even with many concurrent jobs and pipelines running. This also gives us the ability to pull built versions of lattice associated with the commits that produced them, because each new file in S3 also gets a zero-byte file with the long name of the git revision (e.g., a zero-byte file called `924a152ae92372a178bd0fc32b411612b995a6a1` will be stored in S3 so it is possible to know which commit changed the lattice directory.) Invalidating this build cache is as simple as deleting a specific key (or all keys) at s3://molecula-artifact-storage/lattice/<hash> (or s3://molecula-artifact-storage/lattice/*). |
||
|---|---|---|
| .. | ||
| .gitlab-ci.yml | ||
| .perf-able-gitlab-ci.yml | ||
| batch-ci.yml | ||
| Dockerfile | ||