Value error when running

[lfdversluis]

Traceback (most recent call last):
  File "/home/laurensv/pandas_alibaba_analysis.py", line 236, in <module>
    res = grouped_df.parallel_apply(lookahead_newer).to_parquet(os.path.join(output_location_look_ahead, 'output.parquet'), compression='snappy', engine='pyarrow')
  File "/home/laurensv/miniconda3/lib/python3.8/site-packages/pandarallel/pandarallel.py", line 446, in closure
    pool = context.Pool(
  File "/home/laurensv/miniconda3/lib/python3.8/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "/home/laurensv/miniconda3/lib/python3.8/multiprocessing/pool.py", line 205, in __init__
    raise ValueError("Number of processes must be at least 1")
ValueError: Number of processes must be at least 1

It did start with

INFO: Pandarallel will run on 48 workers.
INFO: Pandarallel will use Memory file system to transfer data between the main process and workers.

So something went wrong communicating 48 -> number of processes.

Using version 1.5.2.

[lfdversluis]

https://github.com/nalepae/pandarallel/blob/master/pandarallel/pandarallel.py#L443 I bet this somehow was <= 0

[lfdversluis]

Going by git blame, logic changed in 1.4.7. I will try 1.4.6 and see if that works.

[ztsweet]

hi, does it work successful with 1.4.6?

[till-m]

Closed in favour of #115. Feel free to reopen with a minimal working example if you think this is not the same problem.