dovpanda
dovpanda copied to clipboard
After importing dovpanda, pandas read_csv() doesn't work.
Brief Description
I'm trying to concatenate csv files that start with keyword(example uses A). With dovpanda, the csv files don't seem to be found. I checked the "glob(os.path.join(keyword + '*.csv'))" line by itself, that brings up a list of the correct csv files to concatenate.
System Information
Windows 10 Jupyter Notebook Python 3.6.5 :: Anaconda, Inc.
Minimally Reproducible Code
import pandas as pd
import dovpanda
keyword = str('A')
df = pd.concat(map(pd.read_csv, glob(os.path.join(keyword + '*.csv'))))
display (df)
df.to_csv(path_or_buf=(f"Files {today}.csv"), index=False, encoding='ascii')
Error Messages
SAD PANDA
I'm so sorry, but I crashed on wrong_concat_axis hooks on concat with error descriptor 'union' of 'set' object needs an argument
But you can change that!
Please Report a bug×
Line 5: df = pd.concat(map(pd.read_csv, glob(os.path.join(keyword + '*.csv'))))
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-13-221aac6881a7> in <module>()
3 #CRM File merge
4 keyword = str('A')
----> 5 df = pd.concat(map(pd.read_csv, glob(os.path.join(keyword + '*.csv'))))
6
7 myList = ['enrollment_id','roster_record_id','user_id','email','ssn','identify_as',
~\Anaconda3\lib\site-packages\dovpanda\base.py in run(*args, **kwargs)
154 arguments = self._get_arguments(f, *args, **kwargs)
155 self.run_hints(pres, arguments)
--> 156 ret = f(*args, **kwargs)
157 self.run_hints(posts, ret, arguments)
158 return ret
~\Anaconda3\lib\site-packages\pandas\core\reshape\concat.py in concat(objs, axis, join, join_axes, ignore_index, keys, levels, names, verify_integrity, sort, copy)
226 keys=keys, levels=levels, names=names,
227 verify_integrity=verify_integrity,
--> 228 copy=copy, sort=sort)
229 return op.get_result()
230
~\Anaconda3\lib\site-packages\pandas\core\reshape\concat.py in __init__(self, objs, axis, join, join_axes, keys, levels, names, ignore_index, verify_integrity, copy, sort)
260
261 if len(objs) == 0:
--> 262 raise ValueError('No objects to concatenate')
263
264 if keys is None:
ValueError: No objects to concatenate
Thanks @melissadecastro . Managed to reproduce. This happened because I didn't test for generators in concat. I will work on a fix but for the meantime if you want to continue using dovpanda try:
df = pd.concat(list(map(pd.read_csv, glob(os.path.join(keyword + '*.csv')))))