delta-sharing icon indicating copy to clipboard operation
delta-sharing copied to clipboard

Critical error in python API, killed when subtables are empty

Open lz100 opened this issue 7 months ago • 0 comments

When I use the python API to download some tables, this happed:

xxx/.venv/lib/python3.10/site-packages/delta_sharing/reader.py:123: FutureWarning: 
The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. 
In a future version, this will no longer exclude empty or all-NA columns when determining 
the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
Killed

It happen on this line code in your API:

        merged = pd.concat(
            pdfs,
            axis=0,
            ignore_index=True,
            copy=False,
        )

It seems some of the pfds are empty. Some of other tables worked, but this one had this error. I tried a few times with the same thing. This table is managed by an outside company so I have no idea what's in there. It was killed before I get a chance to see the content.

I was using V1.03 pandas 2.14

lz100 avatar Jan 19 '24 00:01 lz100