dovpanda icon indicating copy to clipboard operation
dovpanda copied to clipboard

After importing dovpanda, pandas read_csv() doesn't work.

Open melissadecastro opened this issue 5 years ago • 1 comments

Brief Description

I'm trying to concatenate csv files that start with keyword(example uses A). With dovpanda, the csv files don't seem to be found. I checked the "glob(os.path.join(keyword + '*.csv'))" line by itself, that brings up a list of the correct csv files to concatenate.

System Information

Windows 10 Jupyter Notebook Python 3.6.5 :: Anaconda, Inc.

Minimally Reproducible Code

import pandas as pd
import dovpanda

keyword = str('A')

df = pd.concat(map(pd.read_csv, glob(os.path.join(keyword + '*.csv'))))
display (df)

df.to_csv(path_or_buf=(f"Files {today}.csv"), index=False, encoding='ascii')

Error Messages

SAD PANDA

I'm so sorry, but I crashed on wrong_concat_axis hooks on concat with error descriptor 'union' of 'set' object needs an argument
But you can change that!
Please Report a bug×
Line 5: df = pd.concat(map(pd.read_csv, glob(os.path.join(keyword + '*.csv'))))
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-13-221aac6881a7> in <module>()
      3 #CRM File merge
      4 keyword = str('A')
----> 5 df = pd.concat(map(pd.read_csv, glob(os.path.join(keyword + '*.csv'))))
      6 
      7 myList = ['enrollment_id','roster_record_id','user_id','email','ssn','identify_as',

~\Anaconda3\lib\site-packages\dovpanda\base.py in run(*args, **kwargs)
    154             arguments = self._get_arguments(f, *args, **kwargs)
    155             self.run_hints(pres, arguments)
--> 156             ret = f(*args, **kwargs)
    157             self.run_hints(posts, ret, arguments)
    158             return ret

~\Anaconda3\lib\site-packages\pandas\core\reshape\concat.py in concat(objs, axis, join, join_axes, ignore_index, keys, levels, names, verify_integrity, sort, copy)
    226                        keys=keys, levels=levels, names=names,
    227                        verify_integrity=verify_integrity,
--> 228                        copy=copy, sort=sort)
    229     return op.get_result()
    230 

~\Anaconda3\lib\site-packages\pandas\core\reshape\concat.py in __init__(self, objs, axis, join, join_axes, keys, levels, names, ignore_index, verify_integrity, copy, sort)
    260 
    261         if len(objs) == 0:
--> 262             raise ValueError('No objects to concatenate')
    263 
    264         if keys is None:

ValueError: No objects to concatenate

melissadecastro avatar Dec 04 '19 21:12 melissadecastro

Thanks @melissadecastro . Managed to reproduce. This happened because I didn't test for generators in concat. I will work on a fix but for the meantime if you want to continue using dovpanda try:

df = pd.concat(list(map(pd.read_csv, glob(os.path.join(keyword + '*.csv')))))

DeanLa avatar Dec 15 '19 16:12 DeanLa