pandarallel TypeError: 'generator' object is not subscriptable error in colab

TypeError: 'generator' object is not subscriptable error in colab - works in VScode

Open agiveon opened this issue 1 year ago • 5 comments

General

Operating System: OSX
Python version: 3.7.13
Pandas version: 1.3.5
Pandarallel version: 1.6.2

Acknowledgement

Issue happens on Colab only. When I use VScode, the problem does not happen

Bug description

I get the error:

100.00%
1 / 1
100.00%
1 / 1
100.00%
1 / 1
100.00%
1 / 1
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[<ipython-input-16-b9ca1a6007f2>](https://localhost:8080/#) in <module>()
      1 data = {'Name': ['Tom', 'Joseph', 'Krish', 'John'], 'Age': [20, 21, 19, 18]}
      2 df = pd.DataFrame(data)
----> 3 df['HalfAge'] = df.parallel_apply(lambda r: r.Age/2,axis=1)

2 frames
[/usr/local/lib/python3.7/dist-packages/pandarallel/core.py](https://localhost:8080/#) in closure(data, user_defined_function, *user_defined_function_args, **user_defined_function_kwargs)
    324             return wrapped_reduce_function(
    325                 (Path(output_file.name) for output_file in output_files),
--> 326                 reduce_extra,
    327             )
    328 

[/usr/local/lib/python3.7/dist-packages/pandarallel/core.py](https://localhost:8080/#) in closure(output_file_paths, extra)
    197         )
    198 
--> 199         return reduce_function(dfs, extra)
    200 
    201     return closure

[/usr/local/lib/python3.7/dist-packages/pandarallel/data_types/dataframe.py](https://localhost:8080/#) in reduce(datas, extra)
     45             datas: Iterable[pd.DataFrame], extra: Dict[str, Any]
     46         ) -> pd.DataFrame:
---> 47             axis = 0 if isinstance(datas[0], pd.Series) else 1 - extra["axis"]
     48             return pd.concat(datas, copy=False, axis=axis)
     49 

TypeError: 'generator' object is not subscriptable

Minimal but working code sample to ease bug fix for `pandarallel` team

!pip install pandarallel
from pandarallel import pandarallel
pandarallel.initialize(progress_bar=True)

import pandas as pd

data = {'Name': ['Tom', 'Joseph', 'Krish', 'John'], 'Age': [20, 21, 19, 18]}
df = pd.DataFrame(data)
df['HalfAge'] = df.parallel_apply(lambda r: r.Age/2,axis=1)

Aug 04 '22 03:08 agiveon

I have same error occured in google colab.

Aug 04 '22 08:08 glassonion1

Hi all,

I think I introduced this bug in a recent PR. I assume rolling back to pandarallel 1.6.1 (i.e. !pip install pandarallel==1.6.1) should work as a temporary fix while I have a look at the issue.

Very sorry about the bug, I tested on regular jupyter notebooks but not in Colab.