pandas

pandas copied to clipboard

Is your feature request related to a problem?

With a group.apply the applied function acts on all columns available to provide a result. With rolling.apply it applies the function to each column individually - it would be great if you could apply it to all the columns as well.

e.g. the following is possible with groupby.apply

df = pd.DataFrame({'A': range(10), 'B': range(10,0,-1)}).reset_index()
df['index'] = df['index'] % 3
df.groupby('index').apply(lambda df: df['A'] + df['B'])

However, you can't do the same equivalent with rolling.apply:

df = pd.DataFrame({'A': range(10), 'B': range(10,0,-1)}).reset_index()
df.rolling(3, on='index').apply(lambda df: (df['A'] * df['B']).sum())

Gives the error: KeyError: 'A'

This is because the apply function is applied to each column separately. The series is passed to the function not the dataframe. This feels inconsistent and annoying if you want to apply a function to the grouped rolling data. It would be great if there were a way to do it 😃

seee https://pandas.pydata.org/pandas-docs/dev/reference/api/pandas.DataFrame.rolling.html#pandas.DataFrame.rolling

this is possible in 1.3 with method='table' and a numba udf.

Hi, thanks - I must of missed that. It took me a while to work out where to add engine='numba'. Can we keep this ticket open and I will add an example to rolling API documentation?

sure happy to have more documentation

Ive added an example to the docs in #40519

Hey there, is there a way to do it without using a numba ufunc? I'm trying to use it with a scikit-learn model and those two libraries are just not compatible at all