Tiansu
Tiansu
> I see. I was under the assumption that PyArrow could completely replace fsspec. But it seems like there are a few use cases where we would prefer fsspec. >...
I will have a look this issue.
A generic question: why we are implementing a custom https://github.com/apache/iceberg-python/blob/dc6d2429aafbffc626cba53aaac3f6198fc37eb3/pyiceberg/io/__init__.py#L320-L329 https://github.com/apache/iceberg-python/blob/dc6d2429aafbffc626cba53aaac3f6198fc37eb3/pyiceberg/io/__init__.py#L290-L300 instead of using `fsspec.core.url_to_fs(file_path)[0]` directly? (On a side note: this looks a bit confusing to me, as why for...
Read my comment [here](https://gist.github.com/kevinjqliu/647808faba256855639e91dd58243082?permalink_comment_id=5175413#gistcomment-5175413) for the cause of the issue. I dont think fixing `SqlCatalog` alone is the proper answer to this bug. The io layer seems to me ill...
@kevinjqliu I think resolving fs at file level should make the API cleaner. We can e.g. if no `file_system` given to `FsspecInputFile` (or similarly `PyarrowInputFile`), then we resolve them at...
I will make a PR according to this: > My preference is resolving fs at the file level. It's more flexible and the performance difference should be negligible. > Another...
Also reading on here: https://arrow.apache.org/docs/python/filesystems.html#using-arrow-filesystems-with-fsspec There might be some opportunity that we can simplify the split between arrow and fsspec file_system.
Reading on table spec, I just realised that there is a field `location` in https://iceberg.apache.org/spec/#table-metadata-fields that specifies a base location of the table. Does Iceberg Java actually allows the split...
I found the same error appears, while the property on the parent class is an abstractmethod. Is this a bug in such case?