Manu Zhang
Manu Zhang
It appears `AmmoniteSparkSession` has been replaced by `NotebookSparkSession` in recent versions. The example in README is not up-to-date and it will be great for this info to be added to...
@MrPowers, @nvander1, do you think it's a good idea to add thie project to https://github.com/fthomas/scala-steward to keep the dependencies up to date ?
Kindle supports "Export Notes" of a book as a CSV file and sending it to your email. It will be great if the plugin can sync a CSV file into...
Currently, during Spark's rewrite data files procedure with bin pack strategy, `SparkSession` is cloned to disable AQE in each `rewriteFiles`. Since a cloned `SparkSession` has its own state, V2SessionCatalog is...
The following application is run on a `local` cluster and `(is,1)(bingo!!,2)(a,1)(good,1)(This,1)(start,,1)` is expected in `gearpump-local-*.log` ```scala val app = StreamApp("dsl", context) val data = "This is a good start, bingo!!...
``` [error] (run-main-0) java.lang.ClassNotFoundException: scala.Int [INFO] [12/15/2018 19:23:44.474] [RemoteActorRefProvider$RemotingTerminator] Shutting down remote daemon. [INFO] [12/15/2018 19:23:44.477] [RemoteActorRefProvider$RemotingTerminator] Remote daemon shut down; proceeding with flushing remote transports. [INFO] [12/15/2018 19:23:44.511] [RemoteActorRefProvider$RemotingTerminator]...
1. ``` [error] /home/travis/build/gearpump/gearpump/streaming/target/java/io/gearpump/streaming/ProcessorDescription.java:5:1: error: illegal combination of modifiers: abstract and static [error] static public abstract boolean canEqual (Object that) ; [error] ``` 2. ``` [error] /home/travis/build/gearpump/gearpump/streaming/target/java/io/gearpump/streaming/transaction/api/CheckpointStoreFactory.java:4:1: error: invalid use...
``` Unable to read classname from bytecode in akka/stream/javadsl/JavaFlowSupport$Flow.class Shading is therefore impossible, so this entry will be skipped. java.lang.IllegalArgumentException: null Unable to read classname from bytecode in akka/stream/javadsl/JavaFlowSupport$Sink.class Shading...
This adds a typed API over Spark ML's `OneHotEncoderEstimator` since Spark 2.3.0
I find all input data are loaded when I just want to print schema with the following code. ```python spark.read \ .option("recordType", "Example") \ .format("tfrecords") \ .load(path) \ .printSchema() ```