incubator-xtable
incubator-xtable copied to clipboard
Add Python wrappers or create interoperability to call the classes/methods using Python
Data engineers quite heavily rely on Python for creating data pipelines. Its important to support the ability to call the Java objects using Python (through Py4J or similar tools/libraries). Right now, users can only use JVM languages out of the box if they were to use OneTable directly i.e. as shown in docker demo.
Why not also non-jvm and then python bindings to that language.
@ion-elgreco any language you are looking for in particular?
Right now, the implementation relies on java since that is the only common language for interacting with tables in the major table formats. We can also consider a deployment model where this code is behind some lightweight server and build an API that users can call. Generating clients in various languages is pretty straightforward so I'm curious what people think of this idea.
Rust in particular and then using pyo3 bindings. We already do this at Delta-RS
I think what would be great is some pyspark examples which show how to invoke onetable