dovpanda icon indicating copy to clipboard operation
dovpanda copied to clipboard

problem in handling empty dataframes

Open ItamarMushkin opened this issue 5 years ago • 2 comments

I'm loading and preprocessing a legitimate dataframe, using a long function, with calls to pymongo and mysql (will try spending the time to make a minimal example if issue is unclear).

I have this line: df2 = pd.concat([v for v in some_cache_df.values()]) if some_cache_df else pd.DataFrame()

It raises an IndexError in line 45 of /dovpanda/core.py, where there's a call to cols = {df.shape[1] for df in objs}

ItamarMushkin avatar Oct 24 '19 08:10 ItamarMushkin

could not reproduce. whaat pandas version? is some_cache_df.values() a pymongo object?

DeanLa avatar Oct 25 '19 22:10 DeanLa

No, is is a dict of dataframes. But I think the problem happens when it's empty, and then going to "pd.DataFrame()". Like I said, it might take effort to create a reproducible example...

ps. sorry for not responding earlier, did not see the notification for the comment

ItamarMushkin avatar Oct 29 '19 06:10 ItamarMushkin