datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Add `deserialize` to `BatchSerializer`

Open jayzhan-synnada opened this issue 9 months ago • 3 comments

Which issue does this PR close?

  • Closes #.

Rationale for this change

we have serialize already, and it would be nice to have another deserialize so we can convert the output of serialize back to RecordBatch

Also, both method are changed to async, since most of the time we prefer async function for serialization in case it takes a long time

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

jayzhan-synnada avatar Mar 25 '25 09:03 jayzhan-synnada

I've two questions: 1) Why don't we use BatchDeserializer to deserialize? 2) Why are we converting them to async as they don't yield at all already?

berkaysynnada avatar Mar 25 '25 14:03 berkaysynnada

BatchDeserializer can do what I want so we don't need this.

Maybe we can fix the code structure, it would be nice that they are close together and clear that we can deserialize and serialize with these 2.

jayzhan-synnada avatar Mar 26 '25 01:03 jayzhan-synnada

Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.

github-actions[bot] avatar Jun 12 '25 02:06 github-actions[bot]