buckaroo
buckaroo copied to clipboard
Figure out how to join polars JLisp commands into a single Lazy block
Checks
- [X] I have checked that this enhancement has not already been requested
How would you categorize this request. You can select multiple if not sure
Developer ergonomics (defaults, error messages), Performance, Developer Experience/CI (feature to make it easier to devlop on Buckaroo), Low Code UI
Enhancement Description
Polars is blazing fast if all operations are collected into a single lazy query. Pandas doesn't get this speed increase and it was simpler to write commands as returning a new dataframe, even if this was slow.
Pseudo Code Implementation
instead of emitting code like
def clean(df):
df = df.with_columns(OP_A)
df = df.with_columns(OP_B)
df = df.with_columns(OP_C)
return df
emit code like this
def clean(df):
return df.with_columns(OP_A, OP_B, OP_C)
Prior Art
n/a