cider icon indicating copy to clipboard operation
cider copied to clipboard

Published 20 hours ago • verified publisher icon clojure-emacs

can't use Clojure 1.12 `add-lib` when `cider-enrich-classpath` set to `t`

Open andreyorst opened this issue 2 months ago • 8 comments

Expected behavior

user> (add-lib 'cheshire/cheshire {:mvn/version "5.13.0"})
[cheshire/cheshire
 com.fasterxml.jackson.core/jackson-core
 com.fasterxml.jackson.dataformat/jackson-dataformat-cbor
 com.fasterxml.jackson.dataformat/jackson-dataformat-smile
 com.google.code.findbugs/jsr305
 com.google.errorprone/error_prone_annotations
 com.google.guava/guava
 com.google.j2objc/j2objc-annotations
 org.checkerframework/checker-compat-qual
 org.codehaus.mojo/animal-sniffer-annotations
 tigris/tigris]

Actual behavior

user> (add-lib 'cheshire/cheshire {:mvn/version "5.13.0"})
Execution error (ExceptionInfo) at clojure.tools.deps.interop/invoke-tool (interop.clj:52).
Could not find artifact com.fasterxml.jackson:jackson-base:pom:2.17.0

1. Unhandled clojure.lang.ExceptionInfo
   Could not find artifact
   com.fasterxml.jackson:jackson-base:pom:2.17.0
   {:via
    [{:type org.eclipse.aether.resolution.ArtifactDescriptorException,
      :message
      "Failed to read artifact descriptor for com.fasterxml.jackson.core:jackson-core:jar:2.17.0",
      :at
      [org.apache.maven.repository.internal.DefaultArtifactDescriptorReader
       loadPom
       "DefaultArtifactDescriptorReader.java"
       302]}
     {:type
      org.apache.maven.model.resolution.UnresolvableModelException,
      :message
      "Could not find artifact com.fasterxml.jackson:jackson-base:pom:2.17.0",
      :at
      [org.apache.maven.repository.internal.DefaultModelResolver
       resolveModel
       "DefaultModelResolver.java"
       176]}
     {:type org.eclipse.aether.resolution.ArtifactResolutionException,
      :message
      "Could not find artifact com.fasterxml.jackson:jackson-base:pom:2.17.0",
      :at
      [org.eclipse.aether.internal.impl.DefaultArtifactResolver
       resolve
       "DefaultArtifactResolver.java"
       431]}
     {:type org.eclipse.aether.transfer.ArtifactNotFoundException,
      :message
      "Could not find artifact com.fasterxml.jackson:jackson-base:pom:2.17.0",
      :at
      [org.eclipse.aether.internal.impl.DefaultArtifactResolver
       resolve
       "DefaultArtifactResolver.java"
       421]}],
    :trace
    [[org.eclipse.aether.internal.impl.DefaultArtifactResolver
      resolve
      "DefaultArtifactResolver.java"
      421]
     [org.eclipse.aether.internal.impl.DefaultArtifactResolver
      resolveArtifacts
      "DefaultArtifactResolver.java"
      235]
     [org.eclipse.aether.internal.impl.DefaultArtifactResolver
      resolveArtifact
      "DefaultArtifactResolver.java"
      212]
     [org.apache.maven.repository.internal.DefaultModelResolver
      resolveModel
      "DefaultModelResolver.java"
      172]
     [org.apache.maven.repository.internal.DefaultModelResolver
      resolveModel
      "DefaultModelResolver.java"
      222]
     [org.apache.maven.model.building.DefaultModelBuilder
      readParentExternally
      "DefaultModelBuilder.java"
      1150]
     [org.apache.maven.model.building.DefaultModelBuilder
      readParent
      "DefaultModelBuilder.java"
      916]
     [org.apache.maven.model.building.DefaultModelBuilder
      build
      "DefaultModelBuilder.java"
      361]
     [org.apache.maven.model.building.DefaultModelBuilder
      build
      "DefaultModelBuilder.java"
      267]
     [org.apache.maven.repository.internal.DefaultArtifactDescriptorReader
      loadPom
      "DefaultArtifactDescriptorReader.java"
      293]
     [org.apache.maven.repository.internal.DefaultArtifactDescriptorReader
      readArtifactDescriptor
      "DefaultArtifactDescriptorReader.java"
      171]
     [org.eclipse.aether.internal.impl.DefaultRepositorySystem
      readArtifactDescriptor
      "DefaultRepositorySystem.java"
      263]
     [clojure.tools.deps.extensions.maven$read_descriptor
      invokeStatic
      "maven.clj"
      115]
     [clojure.tools.deps.extensions.maven$read_descriptor
      invoke
      "maven.clj"
      106]
     [clojure.tools.deps.extensions.maven$eval1054$fn__1055
      invoke
      "maven.clj"
      146]
     [clojure.lang.MultiFn invoke "MultiFn.java" 244]
     [clojure.tools.deps$expand_deps$children_task__615$fn__617$fn__618
      invoke
      "deps.clj"
      406]
     [clojure.lang.AFn applyToHelper "AFn.java" 152]
     [clojure.lang.AFn applyTo "AFn.java" 144]
     [clojure.core$apply invokeStatic "core.clj" 667]
     [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1990]
     [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1990]
     [clojure.lang.RestFn invoke "RestFn.java" 425]
     [clojure.lang.AFn applyToHelper "AFn.java" 156]
     [clojure.lang.RestFn applyTo "RestFn.java" 132]
     [clojure.core$apply invokeStatic "core.clj" 671]
     [clojure.core$bound_fn_STAR_$fn__5818 doInvoke "core.clj" 2020]
     [clojure.lang.RestFn invoke "RestFn.java" 397]
     [clojure.lang.AFn call "AFn.java" 18]
     [java.util.concurrent.FutureTask run "FutureTask.java" 264]
     [java.util.concurrent.ThreadPoolExecutor
      runWorker
      "ThreadPoolExecutor.java"
      1136]
     [java.util.concurrent.ThreadPoolExecutor$Worker
      run
      "ThreadPoolExecutor.java"
      635]
     [java.lang.Thread run "Thread.java" 840]],
    :cause
    "Could not find artifact com.fasterxml.jackson:jackson-base:pom:2.17.0"}
               interop.clj:   52  clojure.tools.deps.interop/invoke-tool
               interop.clj:   15  clojure.tools.deps.interop/invoke-tool
                  deps.clj:   48  clojure.repl.deps/add-libs
                  deps.clj:   57  clojure.repl.deps/add-lib
                  deps.clj:   57  clojure.repl.deps/add-lib
                      REPL:   12  user/eval12491

Steps to reproduce the problem

Given this deps.end:

;; deps.edn
{:deps {org.clojure/clojure {:mvn/version "1.12.0-alpha10"}
        org.clojure/tools.deps {:mvn/version "0.19.1421"}}}
  1. (setq cider-enrich-classpath t)
  2. M-x cider-jack-in
  3. In the REPL (add-lib 'cheshire/cheshire {:mvn/version "5.13.0"})
  4. error

To reproduce the successful behavior:

  1. (setq cider-enrich-classpath nil) or run plain REPL with clojure
  2. M-x cider-jack-in
  3. In the REPL (add-lib 'cheshire/cheshire {:mvn/version "5.13.0"})
  4. success

Environment & Version information

CIDER version information

;; CIDER 1.14.0-snapshot (package: 20240424.1919), nREPL 1.1.1
;; Clojure 1.12.0-alpha10, Java 17.0.9

Lein / Clojure CLI version

Leiningen 2.10.0 on Java 17.0.9 OpenJDK 64-Bit Server VM

Emacs version

GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.41, cairo version 1.17.8) of 2024-04-29

Operating system

Fedora Linux 39.20240421.0 (Silverblue)

JDK distribution

openjdk version "17.0.9" 2023-10-17 OpenJDK Runtime Environment (Red_Hat-17.0.9.0.9-4) (build 17.0.9+9) OpenJDK 64-Bit Server VM (Red_Hat-17.0.9.0.9-4) (build 17.0.9+9, mixed mode, sharing)

andreyorst avatar Apr 30 '24 19:04 andreyorst

Note, Borkdude's deps.add-lib works when enrich-classpath is t

andreyorst avatar Apr 30 '24 19:04 andreyorst

Thanks for the report!

It's a fairly baffling one since Enrich doesn't particularly add or modify Maven stuff. It doesn't add/run code either in your repl JVM.

It does all its work before the JVM is launched. It generates an 'enriched' classpath and then it's done.

Sadly, "Failed to read artifact descriptor for" is an uninformative error message, so you'd have to figure out the real cause.

I'd suggest:

  • run a repl with Enrich enabled
  • locate where com.fasterxml.jackson.core:jackson-core:jar:2.17.0 is in the classpath for that repl
  • check if there's any reason for the artifact descriptor to be unreadable.

Cheers - V

vemv avatar Apr 30 '24 19:04 vemv

I guess there are more problems than that. For instance, at work I regularly experience problems with downloading missing libraries through a proxy if I jack-in and some libraries are not present in the .m2. If I disable the cider-enrich-classpath feature, downloading works properly, and the REPL starts fine. (in this particular issue there's no proxy used)

So I guess maybe the problem lies with how the clojure.sh script that wraps the enrich classpath feature is run by Emacs.

andreyorst avatar Apr 30 '24 19:04 andreyorst

It sounds like Enrich could observe some env vars for proxy stuff, feel free to ellaborate in https://github.com/clojure-emacs/enrich-classpath/issues

(Very commonly apps do that - AFAIK, we cannot guess if we need a proxy / which)

vemv avatar Apr 30 '24 19:04 vemv

locate where com.fasterxml.jackson.core:jackson-core:jar:2.17.0 is in the classpath for that repl

I don't think it's located anywhere yet, as add-libs tries to download it, and fails to find the artifact. Maybe I'm wrong.

Can you elaborate a bit on the steps? I'm a bit lost with this maven stuff.

andreyorst avatar Apr 30 '24 19:04 andreyorst

Hmm, if this stuff isn't too familiar it will probably be quicker if I give it a go.

Please report the proxy issue first, let's get it fixed and see if this one persists.

In the meantime you can disable Enrich (globally or in .dir-locals.el)

vemv avatar Apr 30 '24 19:04 vemv

I went down this rabbit hole and I think the problem is related to how the clojure.sh script calls the clojure command who ends up calling clojure.tools.deps.script.make-classpath2 who will write down the .cpcache/xxxx.basis file which will be used by the tools.deps basis system then used by add-lib.

Here are the differences between how the Clojure script calls clojure.tools.deps.script.make-classpath2/-main vs the same when wrapped by enrich clojure.sh :

("--config-user" "/home/jmonetta/.clojure/deps.edn" "--config-project" "deps.edn" "--basis-file" ".cpcache/405540774.basis"  "--cp-file" ".cpcache/405540774.cp"  "--jvm-file" ".cpcache/405540774.jvm"  "--main-file" ".cpcache/405540774.main"  "--manifest-file" ".cpcache/405540774.manifest" "--config-data" "{:deps {nrepl/nrepl {:mvn/version \"1.1.1\"} cider/cider-nrepl {:mvn/version \"0.47.0\"} refactor-nrepl/refactor-nrepl {:mvn/version \"3.10.0\"}} :aliases {:cider/nrepl {:main-opts [\"-m\" \"nrepl.cmdline\" \"--middleware\" \"[refactor-nrepl.middleware/wrap-refactor,cider.nrepl/cider-middleware]\"]}}}" "-M:dev:cider/nrepl")
("--config-user" ""                                 "--config-project" "deps.edn" "--basis-file" ".cpcache/3864922009.basis" "--cp-file" ".cpcache/3864922009.cp" "--jvm-file" ".cpcache/3864922009.jvm" "--main-file" ".cpcache/3864922009.main" "--manifest-file" ".cpcache/3864922009.manifest" "--skip-cp")

I think the problem is the enrich system is not providing dependencies information to the clojure command since it is going to calculate the classpath itself, but this ends up in the .basis file being written with nil, which then breaks other tools.deps systems.

Also the config user folder looks empty.

jpmonettas avatar May 02 '24 18:05 jpmonettas

Thanks much for the digging into the 🌀🐇!

In Alex words:

yeah, if using -Scp, you're basically turning off the whole "calculate deps and basis" system

Looks like Enrich could do some explicit effort at writing a valid .basis file (probably using the tools.deps API) and passing it as a System property (since I've observed that's what the Clojure CLI commonly does?)

Might be an easy addition.

The --config-user issue looks like a separate one. Might have to do with the tmpdir jumping.

vemv avatar May 02 '24 19:05 vemv