manzt
Anywidget Arrow helpers
Describe the problem
It would be nice to simplify Arrow data transfer. I'd like to make it simple for widget developers using anywidget to take in Arrow data in their widget and transmit it to the frontend.
(It might also be worthwhile to have a discussion about simplifying data transfer in general. Could we have helpers for numpy, pydantic, etc?)
Describe the proposed solution
Maybe have a Python-side and JS-side helper for this?
Is there a way to make this easy? Would we need to require users to remember to depend on apache-arrow or flechette?
Open to thoughts. Is this not worth having in anywidget? Would this be best as a third-party package?
Importance
nice to have
I’m open to small Python-side helpers since that something "host platforms" don't need to implement, but my first instinct is to improve examples/docs/recipes rather than grow the core API. Keeping anywidget minimal is what makes it simple to adopt/host. Maybe an @anywidget/helpers lib could make sense, but I'd like to have a better idea of what could live (beyond aliasing calls to flechette/apache-arrow).
I agree it shouldn't be in the core API, but having something like @anywidget/arrow or @anywidget/table that provides arrow-based helpers could be useful 🤷♂️
I'd like to have a better idea of what could live (beyond aliasing calls to
flechette/apache-arrow).
I'm thinking mostly that it would be nice to have a one-line useTable to get a table in JS from Python. Originally I was thinking it would be nice for anywidget developers to not really need to know about Arrow, to just take in a DataFrame on the Python side and have it magically appear on the JS side.
But I suppose that breaks down because there isn't a canonical JS representation of a table outside of Arrow. So a user has to touch Arrow in the JS side regardless