Add API for generic data directory
This PR implements a minimal generic data directory for arecibo to read/write from. It is similar the the data directory used in Lurk. Besides the immediate need to have some way of managing witness/cross-term vectors for our MSM data analysis, there has been a need for this type of infrastructure for a while.
See, for example, https://github.com/microsoft/Nova/issues/270, which can build upon this PR. There is also https://github.com/lurk-lab/lurk-rs/issues/1086, which may need to move some of Lurk's caching logic into arecibo.
This PR doesn't intend to address all the issues of these PRs, but it would be nice to have the first increment of work streamlined into the codebase, so that we can slowly build up to a robust solution.
Thank you for all the comments! I've written a starting issue here: https://github.com/lurk-lab/arecibo/issues/371. It's more of a set of questions that I have for how to proceed, rather than what I think needs to be done, so I would be happy to discuss and create a plan for what's next.