spiraldb
Simple examples
I'm having a tough time figuring out how to use this library. What I want to do is create some structure, add some data to it, and write it to an arbitrary output sink, not necessarily a file on disk. I'm writing in Rust. The amount of data is large, so output will need to be streaming.
Some simple example code would be helpful. I'm not finding any, either in the docs or in the project.
Pgrx has a really good way of doing this. They have module pgrx-examples, where they have code to do this or that. This code doubles as unit tests.
Hi! Sorry for the long wait.
We have a few examples in some of the subcrates, and we should probably move them into the main vortex crate.
https://github.com/vortex-data/vortex/tree/develop/vortex-datafusion/examples https://github.com/vortex-data/vortex/tree/develop/vortex-ffi/examples
What I want to do is create some structure, add some data to it, and write it to an arbitrary output sink, not necessarily a file on disk. I'm writing in Rust.
I think VortexWriteOptions is what you are looking for? In the vortex-datafusion example we write to a file, but you should be able to write to anything that implements VortexWrite.
The amount of data is large, so output will need to be streaming.
We have AsyncWriteAdapter implement VortexWrite, which means if your sink implements AsyncWrite you should be able to write to it, and then your sink then can output streaming data?
Please let us know if you need something else! (and we'll try to do better with response times but we are a small team)
We definitely need to have better examples though, so I will leave this issue open.
Hello, Recently I'm applying vortex for my individual project. I've used writer API for update, and add few samples of using this in PR https://github.com/vortex-data/vortex/pull/5848
