pranadb
pranadb copied to clipboard
Infinite storage
We should allow for older data (cold-set) to be offloaded to S3 buckets (when running on AWS), this would allow Prana dataset to scale to virtually unlimited sizes. Blocks of data would be read and asynchronously pushed while keeping a hot-set local. This would also allow for smaller local storage to be used, reducing costs. Both Kafka (Confluent) and Red Panda do something similar.