doric icon indicating copy to clipboard operation
doric copied to clipboard

Spark 3.2 new functions

Open eruizalo opened this issue 2 years ago • 0 comments

Look into the following functions if they already are in doric or we have to add them:

  • [ ] def typedlit[T : TypeTag](literal: T): Column
  • [ ] def count_distinct(expr: Column, exprs: Column*): Column
  • [ ] def product(e: Column): Column
  • [x] def sum_distinct(e: Column): Column ✅ covered in #123
  • [ ] def lag(e: Column, offset: Int, defaultValue: Any, ignoreNulls: Boolean): Column
  • [ ] def lead(e: Column, offset: Int, defaultValue: Any, ignoreNulls: Boolean): Column
  • [ ] def bitwise_not(e: Column): Column :interrobang: WIP in #68
  • [x] def shiftleft(e: Column, numBits: Int): Column ✅ covered in #223
  • [x] def shiftright(e: Column, numBits: Int): Column ✅ covered in #223
  • [x] def shiftrightunsigned(e: Column, numBits: Int): Column ✅ covered in #223
  • [ ] def sentences(string: Column, language: Column, country: Column): Column
  • [ ] def sentences(string: Column): Column
  • [ ] def next_day(date: Column, dayOfWeek: Column): Column
  • [ ] def session_window(timeColumn: Column, gapDuration: String): Column
  • [ ] def session_window(timeColumn: Column, gapDuration: Column): Column
  • [ ] def call_udf(udfName: String, cols: Column*): Column

eruizalo avatar Jun 24 '22 11:06 eruizalo