doric
doric copied to clipboard
hablapps
Type safety for spark columns
Related to PR #104 - [ ] Could be used to **doric validations** - [x] Include location #283
- [ ] Types & Custom types (expected: #250) - [x] Dynamic field accessor - [ ] API extension: when builder - [ ] Modularity update - [ ] How...
This issue tracks coverage of Spark types by doric. For each Spark type, doric must allow us: * To create a doric column of the corresponding Scala type * To...
I will add a new section explaining why Doric customs and not Spark User Defined Types (or the differences). This will be probably a FAQ _Originally posted by @eruizalo in...
- [ ] `def cume_dist(): Column` - [ ] `def dense_rank(): Column` - [ ] `def lag(e: Column, offset: Int, defaultValue: Any): Column` - [x] `def lag(columnName: String, offset: Int,...
## Expected Behavior Given a custom `User` type: ```scala case class User(name: String, age: Int) object User { implicit val userst: SparkType.Custom[User, String] = SparkType[String].customType[User](x => { val name ::...
For instance, given the following custom product type: ```scala case class User(name: String, age: Int) object User { implicit val userst: SparkType.Custom[User, String] = SparkType[String].customType[User](x => { val name ::...
Look into the following functions if they already are in doric or we have to add them: - [ ] def typedlit[T : TypeTag](literal: T): Column - [ ] def...
Not sure if this issue worths it, remove the TODO comment if it doesn't - [ ] ArrayType - [ ] ListType - [ ] StructType - [ ] MapType