clean-and-green-philly icon indicating copy to clipboard operation
clean-and-green-philly copied to clipboard

Task: Figure out best practice for caching data in GC

Open cfreedman opened this issue 8 months ago • 0 comments

We still need to make decisions regarding our plans for caching data in between runs on GC, and how that will interact between the VM running our pipeline and eventually the bucket serving our frontend data.

This is a bit nebulous as a ticket, but generally we need a plan for the infrastructure to store all the other auxiliary data involved in data-diffing and long term tracking of source data changes for pipeline and where that's going.

cfreedman avatar May 14 '25 12:05 cfreedman