iceberg-python icon indicating copy to clipboard operation
iceberg-python copied to clipboard

Support execution in Windows using Local File System and NFS

Open rfung777 opened this issue 1 year ago • 4 comments

When running PyIceberg in Windows, pyiceberg.io.pyarrow.PyArrowFileIO.parse_location() incorrectly resolve Windows path, e.g. C:\foobar as a scheme of C and causes Unrecognized filesystem type in URI error later.

If using file:// scheme, the path is also incorrectly resolved to /C:\foobar. see https://stackoverflow.com/a/43925228

rfung777 avatar Aug 04 '24 18:08 rfung777

Thanks for working on this @rfung777 🙌

Ideally, we would also love to have Windows integration tests on Github Actions. WDYT?

Fokko avatar Aug 06 '24 20:08 Fokko

Thanks for reviewing. New to the Windows integration test in GitHub Actions. I will do a bit of research to see how it is done.

ghost avatar Aug 06 '24 22:08 ghost

hi @Fokko , I am not having much luck with setting up the Windows integration tests. It seems to require to install Make and Minio. The changes above seems to be localised in the parser function. Shall we push forward with this?

ghost avatar Aug 11 '24 16:08 ghost

I had an attempt of adding the Windows integration tests. Would you help review again please? Thanks.

ghost avatar Aug 17 '24 19:08 ghost

@rfung777 Sorry, I was out for a while. Let's give it a try 🚀

Fokko avatar Oct 28 '24 14:10 Fokko