vortex icon indicating copy to clipboard operation
vortex copied to clipboard

How to use vortex-data with Arrow Flight gRPC?

Open kesavkolla opened this issue 1 month ago • 2 comments

I'm trying to implement an arrow flight SQL server. I want to know can vortex-data be used as interop between record batches and vortex arrays.

kesavkolla avatar Nov 01 '25 04:11 kesavkolla

I'm assuming this is Vortex Python?

I don't believe there's an easy function exposed at the moment (would be trivial to add) to convert between RecordBatch and Vortex Array.

Longer term, we'd like an equivalent to Flight for Vortex that allows servers to send compressed data all the way to the client, having negotiated which encodings the client supports for decompression.

gatesn avatar Nov 02 '25 02:11 gatesn

I was thinking cross programming languages because of gRPC. Yes if there is better way send vortex arrays directly that is also good. Only thing to keep in mind is Arrow Flight is already implemented by many people. It would be very difficult to adopt different alternative.

If vortex can become gRPC interceptor then it can be used seamlessly

kesavkolla avatar Nov 05 '25 07:11 kesavkolla