python-bigquery-dataframes icon indicating copy to clipboard operation
python-bigquery-dataframes copied to clipboard

rolling functions and merge asof?

Open randomgambit opened this issue 2 years ago • 4 comments

Hello there and thanks for this great package!

I wonder if you plan to implement all the pandas-related rolling functions as well as the merge_asof merge. These are very powerful data manipulation tools. They are available in the spark pandas API, so I wonder if these will be available in bigquery pandas as well.

What do you think? Thanks!

randomgambit avatar Nov 14 '23 10:11 randomgambit

Hi, Thanks for the feature request. We do have support for rolling windows on dataframes and series, see DataFrame.rolling docs. The available aggregations are here: Window docs. Is this sufficient to cover your use case?

As for pandas.merge_asof, this is not yet implemented. I will update this issue when we add this feature.

TrevorBergeron avatar Nov 14 '23 21:11 TrevorBergeron

Thanks @tswast , just to clarify: under the hood what is the package doing exactly? are you translating the pandas methods into corresponding bigquery SQL queries? Or are you doing something else?

Thanks!

randomgambit avatar Dec 05 '23 10:12 randomgambit

@randomgambit Yes, we translate to SQL via Ibis.

tswast avatar Dec 05 '23 19:12 tswast