Java heap space

[iliasku]

[application-akka.actor.default-dispatcher-2] [ActorSystem(application)] Uncaught error from thread [application-akka.actor.default-dispatcher-2] shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:3236) at org.mapdb.DataOutput2.ensureAvail(DataOutput2.java:56) at org.mapdb.DataOutput2.write(DataOutput2.java:74) at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1877) at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1786) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1189) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at org.mapdb.SerializerPojo.serializeUnknownObject(SerializerPojo.java:401) at org.mapdb.SerializerBase.serialize2(SerializerBase.java:531) at org.mapdb.SerializerBase.serialize(SerializerBase.java:150) at org.mapdb.SerializerBase.serialize(SerializerBase.java:94) at org.mapdb.SerializerPojo.serialize(SerializerPojo.java:696) at org.mapdb.BTreeMap$NodeSerializer.serialize(BTreeMap.java:397) at org.mapdb.BTreeMap$NodeSerializer.serialize(BTreeMap.java:288) at org.mapdb.Store.serialize(Store.java:154) at org.mapdb.StoreWAL.update(StoreWAL.java:403) at org.mapdb.Caches$HashTable.update(Caches.java:269) at org.mapdb.EngineWrapper.update(EngineWrapper.java:63) at org.mapdb.BTreeMap.put2(BTreeMap.java:707) at org.mapdb.BTreeMap.put(BTreeMap.java:643) at services.FacialAnalysisService$.appendYesPixels(FacialAnalysisService.scala:102) at models.bot.tasks.FacialAnalysisTask$$anonfun$receive$1$$anonfun$applyOrElse$2$$anonfun$apply$1.apply(FacialAnalysisTask.scala:48) at models.bot.tasks.FacialAnalysisTask$$anonfun$receive$1$$anonfun$applyOrElse$2$$anonfun$apply$1.apply(FacialAnalysisTask.scala:42) at scala.collection.immutable.List.foreach(List.scala:318) at models.bot.tasks.FacialAnalysisTask$$anonfun$receive$1$$anonfun$applyOrElse$2.apply(FacialAnalysisTask.scala:42) at models.bot.tasks.FacialAnalysisTask$$anonfun$receive$1$$anonfun$applyOrElse$2.apply(FacialAnalysisTask.scala:38) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.immutable.List.foreach(List.scala:318) at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) at scala.collection.AbstractTraversable.map(Traversable.scala:105) at models.bot.tasks.FacialAnalysisTask$$anonfun$receive$1.applyOrElse(FacialAnalysisTask.scala:38)

[KevinJMao]

I think this might be due to the size of the covariance matrix defined in MatrixHelpers.scala.

Assuming the bot trained on 30 images that are 200 x 200, the covariance matrix seems like it'd end up being a [40000 x 40000] matrix, which rounds out to just under 12GB of memory.

Turk and Pentland suggested in their original paper a shortcut based on the fact that only a handful of these eigenvectors are useful. Here's an implementation (I haven't had a chance to test/profile due to FB authentication giving me the finger.):

def computeEigenVectors(diffMatrix: Array[Array[Double]]): DoubleMatrix2D = {
  val factory = DoubleFactory2D.dense
  val diffDoubleMatrix = factory.make(diffMatrix.length, diffMatrix(0).length).assign(diffMatrix)

  val shortDoubleMatrix = diffDoubleMatrix.zMult(diffDoubleMatrix, null, 1.0, 0.0, false, true)
  val eigenVectorMatrix = new EigenvalueDecomposition(shortDoubleMatrix).getV
  diffDoubleMatrix.zMult(eigenVectorMatrix, null)
}

[aroyer]

might be related https://github.com/crockpotveggies/tinderbox/issues/74

[AlecZadikian9001]

I'm still having this problem. Tried in OS X and Ubuntu Server, both running Java 1.8.