MemoryError in Metaspace

[kephale]

I'm getting a Caused by: java.lang.OutOfMemoryError: Metaspace from sessions where I create too many clisk.live trees (4k trees of depth <10).

Relevant lines from stack trace (using the 0.11.0 release) at clisk.util$expression_info_internal.invoke(util.clj:32) at clisk.node$node_info.invoke(node.clj:435) at clisk.node$validate.invoke(node.clj:451) at clisk.core$image.doInvoke(core.clj:73)

Edit: I know the simple workaround is increasing metaspace, but I'm wondering if there is an alternative. Ideally this will run for ~100k-1m trees (or more).

[mikera]

Might be dependent on Clojure / Java version? What are you using?

Also it may be possible to strip out the validate call altogether for runtime use

[kephale]

Clojure 1.7 (I'll try 1.8 on the next run) Oracle Java 1.8.0_112-b16

Aha, interesting, I'll try removing the validate calls as well!

Thank you!

[mikera]

Oh one other thing to think about - be careful not to hold onto to many references to the compiled nodes, they basically all require generation of custom classes which might be what is exhausting your metaspace. A few should be OK but this was never designed to handle hundred of thousands of compiled classes at once.

[kephale]

For the record, I believe using an alternative version of image without validate solved it!

[mikera]

Cool thanks for confirming it worked!

[mikera]

Leaving open as a reminder in case we want to optimise this in the future