ecosystem

ecosystem copied to clipboard

Caused by: java.lang.NullPointerException at org.tensorflow.spark.datasources.tfrecords.serde.DefaultTfRecordRowEncoder$.$anonfun$encodeFeature$5(DefaultTfRecordRowEncoder.scala:136) at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238) at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36) at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33) at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198) at scala.collection.TraversableLike.map(TraversableLike.scala:238) at scala.collection.TraversableLike.map$(TraversableLike.scala:231) at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:198) at org.tensorflow.spark.datasources.tfrecords.serde.DefaultTfRecordRowEncoder$.encodeFeature(DefaultTfRecordRowEncoder.scala:136) at org.tensorflow.spark.datasources.tfrecords.serde.DefaultTfRecordRowEncoder$.$anonfun$encodeExample$1(DefaultTfRecordRowEncoder.scala:64) at scala.collection.immutable.List.foreach(List.scala:392) at org.tensorflow.spark.datasources.tfrecords.serde.DefaultTfRecordRowEncoder$.encodeExample(DefaultTfRecordRowEncoder.scala:61) at org.tensorflow.spark.datasources.tfrecords.DefaultSource.$anonfun$createRelation$3(DefaultSource.scala:61) at...

SoloBean

Build new Spark-Tensorflow-Connector for scala 2.12

5

Spark has released 3.x which built with scala 2.12, but th newest version of Spark-Tensorflow-Connector is built with scala 2.11 which means not compatiable with spark 3.x. [Maven repository](https://mvnrepository.com/artifact/org.tensorflow/spark-tensorflow-connector_2.11) is...

wuxianxingkong

BytesList with length 0 or 1 is inferred to have StringType instead of ArrayType

1

If `BytesList` in TFRecords has always length of 0 or 1, then the feature is inferred to have `StringType` instead of `ArrayType`. Is there a reason for this behavior? With...

jukujala

[spark-tensorflow-connector] Cannot read multiple TFRecord files

2

Using `spark.read.format("tfrecord").load("path/to/one-file.tfrecord")`, works. How do I read multiple directories with tfrecords in each? I have tried: `spark.read.format("tfrecord").load(paths: _*)`, where paths is an array of paths. `spark.read.format("tfrecord").load(path)`, where path is a...

prasannaVijay

Bumps hadoop-common from 2.10.1 to 3.2.3. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a...

dependabot[bot]

ResNet56 model (with custom training loop) variables are created on parameter server jobs, and updated by workers. Evaluation is done using a dedicated job which uses the checkpoints saved during...

selcukgun

Failing to save the model pickle. and how to check for the predictions?

1

Hi Team , I am trying to train the Tensorflow LSTM model over the Spark after the epoch training, it is failing with the below error:- ========================== Doing CPU training......

vizzyno1

Is it possible to run parameter server based training with spark tensorflow distributor library? or if there are any plans for this functionality. https://www.tensorflow.org/tutorials/distribute/parameter_server_training

ronnygeo

Hi, I have read the module of “kubeflow” and “spark-tensorflow-distributor” in your Github project, and the above “Spark cluster deep learning” projects are very fascinating to me. Now, I am...

JenniferYingyiWu2020

Functionality to save the trained model

4

Is there a way to save the trained model or its weights when using MirroredStrategyRunner?

ShayaNovelis