polars icon indicating copy to clipboard operation
polars copied to clipboard

clip method's param doesn't support pyexpr

Open yutiansut opened this issue 3 years ago • 2 comments

Describe your feature request

Please describe the behavior you want and the motivation. Please also provide examples of how polars would be used if your feature request were added.

If you're not sure what to write here, then try imagining what the ideal documentation of your new feature would look like Then try to write it.

when using the df.groupby().agg , all method should be Expr, however, when we try to get a mad

def mad(x:pl.Expr):
      median = x.median()
      MAD = (x - median).abs().median()
      threshold_up = median + 3 * 1.483 * MAD
      threshold_down = median - 3 * 1.483 * MAD
      return x.clip(threshold_down , threshold_up)

the param is also a Expr

yutiansut avatar Mar 27 '22 19:03 yutiansut

this code is ok

pl.apply(pl.col('xxxxxx'), lambda x: mad(x[0])

yutiansut avatar Mar 27 '22 20:03 yutiansut

Reopening, as clip not supporting an Expression for the min and max parameters is requested more often, see https://stackoverflow.com/questions/73528449/polars-how-do-i-best-clip-a-numerical-column-to-a-certain-quantile

zundertj avatar Aug 29 '22 14:08 zundertj

I would really love to see this feature.

akdienes avatar Feb 09 '23 22:02 akdienes

I'm using this workaround in the meantime (as a method of an register_expr_namespace-annotated class)

    def clip(self, lower: float | pl.Expr, upper: float | pl.Expr) -> pl.Expr:
        return (
            pl.when(self._expr < lower)
            .then(lower)
            .when(self._expr > upper)
            .then(upper)
            .otherwise(self._expr)
        )

(Not sure if I want to really accept general non-scalar Expr but it keeps my Python typing happy...)

tmct avatar Sep 07 '23 13:09 tmct