frameless

frameless copied to clipboard

Support for UDAF

3

Currently we have support for UDF only. UDAF are really useful and it will be nice if we can support them. Relevant scala [docs](http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.expressions.UserDefinedAggregateFunction) and [this](https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.expressions.Aggregator). The second seems to...

imarios

Named intermediate Datasets

4

```scala case class Foo(bar: Int, baz: String, bal: Boolean) val ds: TypedDataset[Foo] ``` This is nicely *named* and *typed*! However, after a select, the names are completely lost: ```scala val...

OlivierBlanvillain

Unexpected behavior of Encoding of nested Options

8

```scala > val f: Option[Option[Option[Int]]] = None > val d = TypedDataset.create( f :: Nil ) > println( d.collect().run() ) WrappedArray(Some(Some(None))) ``` expected is `WrappedArray(None)`

imarios

project idea: frameless-scalacheck

1

Holden Karau has a project, https://github.com/holdenk/spark-testing-base, that provides scalacheck `Gen`s and `assert` helpers for `RDD`, `Dataframe`, and `Dataset`. It would be cool if there was a frameless-scalacheck sub-project with a...

tscholak

Benchmark compilation time

1

Would be nice to use [shapeless.test.compileTime](https://github.com/milessabin/shapeless/blob/ab081796c183530efdd8b29dab8fee1fee7c61f9/core/src/main/scala/shapeless/test/compiletime.scala) or something to check that we don't accidentally introduce regression in compilation time.

OlivierBlanvillain

Updates [org.scoverage:sbt-scoverage](https://github.com/scoverage/sbt-scoverage) from 2.0.0 to 2.0.5. [GitHub Release Notes](https://github.com/scoverage/sbt-scoverage/releases/tag/v2.0.5) - [Version Diff](https://github.com/scoverage/sbt-scoverage/compare/v2.0.0...v2.0.5) I'll automatically update this PR to resolve conflicts as long as you don't change it yourself. If you'd...

typelevel-steward[bot]

Updates [org.scala-lang:scala-library](https://github.com/scala/scala) from 2.13.8 to 2.13.9. [GitHub Release Notes](https://github.com/scala/scala/releases/tag/v2.13.9) - [Version Diff](https://github.com/scala/scala/compare/v2.13.8...v2.13.9) I'll automatically update this PR to resolve conflicts as long as you don't change it yourself. If you'd...

typelevel-steward[bot]

Update sbt-typelevel-ci-release, ... to 0.4.15

1

Updates * [org.typelevel:sbt-typelevel-ci-release](https://github.com/typelevel/sbt-typelevel) * [org.typelevel:sbt-typelevel-site](https://github.com/typelevel/sbt-typelevel) from 0.4.13 to 0.4.15. [GitHub Release Notes](https://github.com/typelevel/sbt-typelevel/releases/tag/v0.4.15) - [Version Diff](https://github.com/typelevel/sbt-typelevel/compare/v0.4.13...v0.4.15) I'll automatically update this PR to resolve conflicts as long as you don't change it...

typelevel-steward[bot]

Remove `Lazy` for Scala 2.13

11

`Lazy` is not only not required for Scala 2.13, but it raised warnings (see https://github.com/tpolecat/doobie/issues/1513 ). Removing it only for Scala 2.13 using cross version compatiblity.

cchantep

Updates [org.scala-lang:scala-library](https://github.com/scala/scala) from 2.12.16 to 2.12.17. [GitHub Release Notes](https://github.com/scala/scala/releases/tag/v2.12.17) - [Version Diff](https://github.com/scala/scala/compare/v2.12.16...v2.12.17) I'll automatically update this PR to resolve conflicts as long as you don't change it yourself. If you'd...

typelevel-steward[bot]