Consider supporting the R package `narrow`.

[lgautier]

narrow is a lighter weight wrapper of Arrow for R than arrow. rpy2-arrow could give the option to use it instead of arrow.

[paleolimbot]

Nice! FWIW, the narrow prototype will get moved to apache/arrow-nanoarrow/r (and be called nanoarrow) in the next month or so.

[lgautier]

Noted. Thanks. (@eddelbuettel who pointed out narrow to me and asked whether rpy2 could use it).

[eddelbuettel]

That's good news! I have been relying on narrow for some time in CI -- as @paleolimbot knows from the issues I filed six months ago -- and considered forking / uploading it myself. I would be happy to look at nanoarrow and let 'team arrow' take care of that.

The C interface is great. Stable (minimal) helpers on top is even better.

[paleolimbot]

I think you're going to like it! There is a well-tested and easily vendorable C library underneath plus I'm planning to expose more helpers in the header :)

[eddelbuettel]

@paleolimbot One question, if I may. In the issue linked above you mention Table and RecordBatch. We are interested in assembling RecordBatch objects from columns, which narrow supported. I don't quite see anything in nanoarrow -- is calling arrow::RecordBatch$import_from_c() the prescribed (and only) approach? Are you planning to add more functionality?

[paleolimbot]

Yes, that will be supported in nanoarrow! Right now I'm spending my development time making sure that the conversion in the other direction has full type coverage and is on par with Arrow's conversions (https://github.com/apache/arrow-nanoarrow/pull/65). Conversion in the other direction - both the "build by buffer/children" and "convert from R vector" is the next PR.

[eddelbuettel]

Great, thank you for the update. Didn't quite know where to ask otherwise.

[lgautier]

T

narrow is a lighter weight wrapper of Arrow for R than arrow. rpy2-arrow could give the option to use it instead of arrow.

The repos for narrow has moved to https://github.com/apache/arrow-nanoarrow

[eddelbuettel]

Nice to see you got back to this too. I was just looking into rpy2-arrow the other day and re-installed it here.

ryp is interesting too, if more minimal.

[lgautier]

Nice to see you got back to this too. I was just looking into rpy2-arrow the other day and re-installed it here.

ryp is interesting too, if more minimal.

Thanks. From a quick glance there is seems be similarity, inspiration, or borrowing from rpy2's rinterface layer. I believe that it could have been implemented using that lower-level "rinterface" layer in rpy2 as an alternative to the "robjects" layer, and a couple of MRs contributed to fix/improve that lower-level interface.

Speaking of low-level interface and high-level interface. I am rolling out changes that have been in my TODO stack for a long time:

• make rpy2 a namespace umbrella package
• split rpy2's low-level (rinterface) and high-level (robjects) into packages installing within that umbrella namespace. robjects depends on rinterface, but rinterface can be installed or used without robjects (and more minimal interface be built out of it)

More about this at https://github.com/rpy2/rpy2/issues/1130.