Austin in the wrong profile causes AOT crash

[sgrove]

mea culpa for leaving [com.cemerick/austin "0.1.1"] in the general plugins section of my project.clj, but it caused some pretty difficult to track down problems with AOT (specifically with the clojure.tools.reader.edn reader-types class). Not sure how to fix this more generally - austin shouldn't be involved in production cljs, since it wouldn't work anyway, but it would be nice if it errored-out or warned when used in production profiles.

Here is the example output that I had when tracking down the issue:

$ lein clean; DEBUG=true lein with-profile +uberjar uberjar; java -jar target/jida-0.1.1-standalone.jar 6543
Leiningen's classpath: :/Users/sgrove/.lein/self-installs/leiningen-2.3.3-standalone.jar
Applying task with-profile to (+uberjar uberjar)
Performing task 'uberjar' with profile(s): 'default,uberjar'
Applying task uberjar to nil
Applying task javac to nil
Running javac with [@/var/folders/39/47_0ln1n3q31k_p7zf4cq8l80000gq/T/.leiningen-cmdline1966769186414931876.tmp]
Applying task compile to nil
Compiling jida.datomic
Compiling jida.db
Compiling jida.models.query
Compiling jida.models.repo
Compiling jida.queries
Compiling jida.queue
Compiling jida.routes.api
Compiling jida.routes.content
Compiling jida.server
Compiling jida.support.cors
Compiling jida.views.common
Compiling jida.views.welcome
Warning: skipped duplicate file: config.clj
Created /Users/sgrove/code/clojure/jida/target/jida-0.1.1.jar
Including jida-0.1.1.jar
Including jsr305-1.3.9.jar
Including guava-14.0.1.jar
Including core.match-0.2.0.jar
Including jetty-io-7.6.8.v20121106.jar
Including args4j-2.0.16.jar
Including core.async-0.1.242.0-44b1e3-alpha.jar
Including servlet-api-2.5.jar
Including log4j-over-slf4j-1.7.5.jar
Including jcl-over-slf4j-1.7.5.jar
Including netty-3.6.6.Final.jar
Including javax.servlet-2.5.0.v201103041518.jar
Including slf4j-api-1.7.5.jar
Including slf4j-nop-1.7.5.jar
Including closure-compiler-v20130603.jar
Including tools.trace-0.7.6.jar
Including jetty-continuation-7.6.8.v20121106.jar
Including aws-java-sdk-1.5.5.jar
Including ring-codec-1.0.0.jar
Including data.json-0.2.3.jar
Including clout-1.0.1.jar
Including jackson-mapper-asl-1.8.9.jar
Including core.incubator-0.1.0.jar
Including jackson-dataformat-smile-2.2.1.jar
Including jackson-core-2.2.1.jar
Including korma-0.3.0-RC4.jar
Including java.jdbc-0.3.0-alpha5.jar
Including protobuf-java-2.4.1.jar
Including piggieback-0.1.0.jar
Including clojurescript.test-0.1.0.jar
Including ring-core-1.2.0.jar
Including dommy-0.1.2.jar
Including tools.macro-0.1.0.jar
Including ring-jetty-adapter-1.2.0.jar
Including tomcat-jdbc-7.0.27.jar
Including google-closure-library-0.0-20130212-95c19e7f0f5f.jar
Including cheshire-5.2.0.jar
Including clj-time-0.4.4.jar
Including austin-0.1.1.jar
Including ring-json-0.2.0.jar
Including commons-io-2.4.jar
Including crate-0.2.3.jar
Including tools.logging-0.2.6.jar
Including commons-codec-1.3.jar
Including cljs-ajax-0.2.1.jar
Including clojurescript-0.0-1934.jar
Including clojure-1.5.1.jar
Including cljs-test-0.0.6.jar
Including tools.cli-0.2.2.jar
Including rhino-1.7R4.jar
Including hiccup-1.0.4.jar
Including tigris-0.1.1.jar
Including compojure-1.1.5.jar
Including joda-time-2.1.jar
Including commons-compiler-jdk-2.6.1.jar
Including google-closure-library-third-party-0.0-2029-2.jar
Including lucene-core-3.3.0.jar
Including jetty-http-7.6.8.v20121106.jar
Including datomic-free-0.8.4218.jar
Including tools.nrepl-0.2.3.jar
Including hornetq-core-2.2.21.Final.jar
Including jackson-core-asl-1.8.9.jar
Including commons-fileupload-1.3.jar
Including jetty-util-7.6.8.v20121106.jar
Including fressian-0.6.3.jar
Including json-20090211.jar
Including h2-1.3.171.jar
Including commons-compiler-2.6.1.jar
Including ring-servlet-1.2.0.jar
Including tomcat-juli-7.0.27.jar
Including jul-to-slf4j-1.7.5.jar
Including c3p0-0.9.1.2.jar
Including postgresql-9.1-901-1.jdbc4.jar
Including tools.reader-0.7.10.jar
Including jetty-server-7.6.8.v20121106.jar
Including carica-1.0.3.jar
Created /Users/sgrove/code/clojure/jida/target/jida-0.1.1-standalone.jar
Exception in thread "main" java.lang.NoClassDefFoundError: clojure/tools/reader/reader_types/Reader
          at clojure.tools.reader.edn__init.load(Unknown Source)
          at clojure.tools.reader.edn__init.<clinit>(Unknown Source)
          at java.lang.Class.forName0(Native Method)
          at java.lang.Class.forName(Class.java:266)
          at clojure.lang.RT.loadClassForName(RT.java:2098)
          at clojure.lang.RT.load(RT.java:430)
          at clojure.lang.RT.load(RT.java:411)
          at clojure.core$load$fn__5018.invoke(core.clj:5530)
          at clojure.core$load.doInvoke(core.clj:5529)
          at clojure.lang.RestFn.invoke(RestFn.java:408)
          at clojure.core$load_one.invoke(core.clj:5336)
          at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
          at clojure.core$load_lib.doInvoke(core.clj:5374)
          at clojure.lang.RestFn.applyTo(RestFn.java:142)
          at clojure.core$apply.invoke(core.clj:619)
          at clojure.core$load_libs.doInvoke(core.clj:5413)
          at clojure.lang.RestFn.applyTo(RestFn.java:137)
          at clojure.core$apply.invoke(core.clj:619)
          at clojure.core$require.doInvoke(core.clj:5496)
          at clojure.lang.RestFn.invoke(RestFn.java:421)
          at jida.datomic$loading__4910__auto__.invoke(datomic.clj:1)
          at jida.datomic__init.load(Unknown Source)
          at jida.datomic__init.<clinit>(Unknown Source)
          at java.lang.Class.forName0(Native Method)
          at java.lang.Class.forName(Class.java:266)
          at clojure.lang.RT.loadClassForName(RT.java:2098)
          at clojure.lang.RT.load(RT.java:430)
          at clojure.lang.RT.load(RT.java:411)
          at clojure.core$load$fn__5018.invoke(core.clj:5530)
          at clojure.core$load.doInvoke(core.clj:5529)
          at clojure.lang.RestFn.invoke(RestFn.java:408)
          at clojure.core$load_one.invoke(core.clj:5336)
          at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
          at clojure.core$load_lib.doInvoke(core.clj:5374)
          at clojure.lang.RestFn.applyTo(RestFn.java:142)
          at clojure.core$apply.invoke(core.clj:619)
          at clojure.core$load_libs.doInvoke(core.clj:5413)
          at clojure.lang.RestFn.applyTo(RestFn.java:137)
          at clojure.core$apply.invoke(core.clj:619)
          at clojure.core$require.doInvoke(core.clj:5496)
          at clojure.lang.RestFn.invoke(RestFn.java:619)
          at jida.server$loading__4910__auto__.invoke(server.clj:1)
          at jida.server__init.load(Unknown Source)
          at jida.server__init.<clinit>(Unknown Source)
          at java.lang.Class.forName0(Native Method)
          at java.lang.Class.forName(Class.java:266)
          at clojure.lang.RT.loadClassForName(RT.java:2098)
          at clojure.lang.RT.load(RT.java:430)
          at clojure.lang.RT.load(RT.java:411)
          at clojure.core$load$fn__5018.invoke(core.clj:5530)
          at clojure.core$load.doInvoke(core.clj:5529)
          at clojure.lang.RestFn.invoke(RestFn.java:408)
          at clojure.lang.Var.invoke(Var.java:415)
          at jida.server.<clinit>(Unknown Source)
Caused by: java.lang.ClassNotFoundException: clojure.tools.reader.reader_types.Reader
       at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
       at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
       at java.security.AccessController.doPrivileged(Native Method)
       at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
       at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
       ... 54 more

As a question though, how do you go about disabling austin when compiling for production? Do you simple comment-out the line in the :require ns-form?

[cemerick]

Maybe I missed something scrolling through the output, but how is Austin affecting AOT? Not to say I doubt that Austin is to blame, I just can't think of a way, and I don't see a mention of it in the stack, etc.

[danenania]

I ran into the same issue in the early stages of a ring/compojure/http-kit app.

I added austin and everything was working fine, but then I ran a lein do clean and after this I started getting the same kind of stack trace that sgrove posted ending in Exception in thread "main" java.lang.NoClassDefFoundError: clojure/tools/reader/reader_types/Reader

I didn't realize that it had anything to do with austin until I asked for help in IRC, so I began selectively removing bits and pieces to see where the culprit was, and I narrowed it down to any calls involving middleware. Removing austin as a plugin fixed the issue.

[danenania]

I should also note that I had austin in my :dev profile.

[cemerick]

Oh, I'll bet there's a version mismatch going on. What revs of ClojureScript, lein-cljsbuild, and austin are you using?

[danenania]

austin: 0.1.3 clojurescript: 0.0-2030 lein-cljsbuild: 0.3.3

[sritchie]

Same issue here. Didn't realize this was Austin's doing.

[sritchie]

same here, with 1.0.0 lein-cljsbuild and cljs 2016. austin 0.1.3. Excluding org.clojurescript/clojurescript doesn't help either.

[sritchie]

My problem goes away if I run my uberjar with an explicit main class, instead of

java -cp myjar.jar clojure.main -m paddleguru.server

That was my mistake, as doing it this way double-compiles, causing issues with protocols.

My guess is that Austin includes some dep, maybe piggieback, that depends on cljs and has a (:gen-class) in the header and is compiling the tools.reader protocols.

[sritchie]

I managed to get around this by moving the plugin to dev, then using these two macros to add my dev calls to my code shared with production:

(defn try-require [sym]
    (try (require sym)
         (catch Throwable _
           (println "Namespace not available in current mode!" sym))))

  (defmacro maybe-resolve [ns method]
    `(when-let [n# (find-ns (quote ~ns))]
       (when-let [m# (ns-resolve n# (quote ~method))]
         @m#)))

[bellkev]

I'm having the same issue in this project: https://github.com/bellkev/dacom/tree/2a352a4da42993c046bc018a5dbf1b1080e959d4

Moving the plugin to the dev profile was a sufficient workaround for me.

[LiFlash]

+1 Additionally I had to move the aot to my production profile. (Since I'm new to clojure I'm not sure if this makes sense)

[sritchie]

@LiFlash yup, that does make sense. I have it in my uberjar profile.

[LiFlash]

@sritchie Thanks for the info. Hopefully I will understand what it is about (besides not delivering the source code), soon ;)

[cemerick]

I have a sneaking suspicion this is related to #37…

Open to any and all ideas as to what's going on.

[blaenk]

Moving it to [:dev :plugins] does solve this issue for me, however, how do you guys go about building an uberjar of the project with :aot :all? To serve the javascript for austin I have:

[cemerick.austin.repls :refer [browser-connected-repl-js]]

but since austin is only in :dev, the uberjar build fails since it can't find the symbol.

[sritchie]

I do this for Weasel (same pattern worked for Austin):

(ns paddleguru.repl
  "Helpers for interacting with Austin: https://github.com/cemerick/austin"
  (:require [paddleguru.util :as u]))

(u/try-require 'cemerick.piggieback)
(u/try-require 'weasel.repl.websocket)

;; Use these to get the REPL running locally, connected to our live
;; server instance. For this to work you'll need to have started the
;; app from the repl itself.
(defn start! []
  (when-let [cljs-repl (u/maybe-resolve cemerick.piggieback cljs-repl)]
    (when-let [repl-env (u/maybe-resolve weasel.repl.websocket repl-env)]
      (cljs-repl :repl-env (repl-env)))))

Here are try-require and friends:

(s/defn try-require :- nil
    [sym :- Symbol]
    (try (require sym)
         (catch Throwable _
           (println "Namespace not available in current mode!" sym))))

  (defmacro maybe-resolve [ns method]
    `(when-let [n# (find-ns (quote ~ns))]
       (when-let [m# (ns-resolve n# (quote ~method))]
         @m#)))

That way I can have code that injects the repl snippet that works in dev mode, but just returns nil in prod.

[arohner]

Complete speculation, but this might be related to: http://dev.clojure.org/jira/browse/CLJ-1544

I see CLJ-1544 when a namespace containing a protocol manages to get require'd before AOT starts.

[Bronsa]

I have investigated http://dev.clojure.org/jira/browse/CLJ-1544 today and I can confirm this is the root cause for this ticket.

[cemerick]

Thanks so much for tracking this down, @Bronsa and @arohner, much appreciated!