koalas icon indicating copy to clipboard operation
koalas copied to clipboard

Koalas.read_*() methods replace path(type:str)

Open shlokc9 opened this issue 5 years ago • 4 comments

Replacing path types accepted by all read_*() methods supported under Koalas from path(type: str) to path(type: filepath_or_buffer: Union[str, pathlib.Path, IO[~AnyStr]]) equivalent to all Pandas.read_*() methods

shlokc9 avatar May 09 '20 16:05 shlokc9

I second the support of pathlib.Path. This is easy to implement and seems idiomatic for python 3.

ikravets avatar May 10 '20 19:05 ikravets

It doesn't make much sense to support buffer which lives in single machine because Koalas targets to scale out the dataset. I think we should explicitly don't support it.

HyukjinKwon avatar May 11 '20 04:05 HyukjinKwon

@ikravets @HyukjinKwon I am working on project that show case Pandas as well as Koalas functionalities together. So long story short; I need to pass an IO type object to Koalas.read_csv() and Koalas.read_excel() methods. I hope it is not as difficult to implement. If not can I know an ETA on this update?

shlokc9 avatar May 12 '20 11:05 shlokc9

@HyukjinKwon , Should we add a note to our documents that we don't want to support this, and close this issue ?

itholic avatar Aug 09 '21 13:08 itholic