jgo icon indicating copy to clipboard operation
jgo copied to clipboard
verified publisher icon scijava

Using jgo within a DockerFile and with conda

Open tischi opened this issue 2 years ago • 6 comments

@ctrueden first of all thanks a lot for providing this tool. awesome!

As you see from the issue title I am quite ambitious and have some little issues.

Not sure you can help but these are the Docker commands:

ENV JGO_CACHE_DIR=/opt/jgo-cache
WORKDIR /opt
RUN mkdir /opt/jgo-cache && \
  source /opt/conda/etc/profile.d/conda.sh && \
  conda create --yes --name mobie-java -c conda-forge python=3.9 mobie && \
  echo $'#!/usr/bin/env -S bash --noprofile --norc\nsource /opt/conda/etc/profile.d/conda.sh\nconda activate mobie-java\nexport JGO_CACHE_DIR=/opt/jgo-cache\nmobie "$@"' > /usr/local/bin/mobie && \
  chmod +x /usr/local/bin/mobie && \
  chmod 777 $JGO_CACHE_DIR && \
  /usr/local/bin/mobie && \    # <== I am trying to launch mobie once to download all dependencies, this is causing the error
  echo Finished installing mobie && \
  date

And that gives me this error

Traceback (most recent call last):
  File "/opt/conda/envs/mobie-java/bin/mobie", line 10, in <module>
    sys.exit(launch_mobie())
  File "/opt/conda/envs/mobie-java/lib/python3.9/site-packages/mobie/__init__.py", line 17, in launch_mobie
    return jgo.util.main_from_endpoint(
  File "/opt/conda/envs/mobie-java/lib/python3.9/site-packages/jgo/util.py", line 95, in main_from_endpoint
    main(argv=argv)
  File "/opt/conda/envs/mobie-java/lib/python3.9/site-packages/jgo/jgo.py", line 406, in _jgo_main
    completed_process = run(parser, argv=argv, stdout=stdout, stderr=stderr)
  File "/opt/conda/envs/mobie-java/lib/python3.9/site-packages/jgo/jgo.py", line 769, in run
    primary_endpoint, workspace = resolve_dependencies(
  File "/opt/conda/envs/mobie-java/lib/python3.9/site-packages/jgo/jgo.py", line 701, in resolve_dependencies
    link(
  File "/opt/conda/envs/mobie-java/lib/python3.9/site-packages/jgo/jgo.py", line 215, in link
    raise e
  File "/opt/conda/envs/mobie-java/lib/python3.9/site-packages/jgo/jgo.py", line 212, in link
    return link(source=source, link_name=link_name, link_type="hard")
  File "/opt/conda/envs/mobie-java/lib/python3.9/site-packages/jgo/jgo.py", line 207, in link
    return os.link(source, link_name)
FileNotFoundError: [Errno 2] No such file or directory: '/home/jovyan/.m2/repository/org/jruby/joni/joni/2.1.29/joni-2.1.29.jar' -> '/opt/jgo-cache/org.embl.mobie/mobie-viewer-fiji/3.0.12/32da8e8da4470d39be2214f164c889f873609a9e59ff5ff0db64668c96b510b9/joni-2.1.29.jar'
error building image: error building stage: failed to execute command: waiting for process to exit: exit status 1

/home/jovyan is the ~ home dir of that build environment.

If you have any idea what this error could be due to it would be very helpful!

Here again just the error: FileNotFoundError: [Errno 2] No such file or directory: '/home/jovyan/.m2/repository/org/jruby/joni/joni/2.1.29/joni-2.1.29.jar' -> '/opt/jgo-cache/org.embl.mobie/mobie-viewer-fiji/3.0.12/32da8e8da4470d39be2214f164c889f873609a9e59ff5ff0db64668c96b510b9/joni-2.1.29.jar'

In fact we were not even sure what the -> means in this context.

tischi avatar Jan 31 '23 15:01 tischi

maybe related: https://github.com/conda-forge/jgo-feedstock/issues/21#issue-1564281520

tischi avatar Jan 31 '23 15:01 tischi

Here a simplified version of that error without changing the JGO_CACHE_DIR

WORKDIR /opt
RUN source /opt/conda/etc/profile.d/conda.sh && \
  conda create --yes --name mobie-java -c conda-forge python=3.9 mobie && \
  echo $'#!/usr/bin/env -S bash --noprofile --norc\nsource /opt/conda/etc/profile.d/conda.sh\nconda activate mobie-java\nmobie "$@"' > /usr/local/bin/mobie && \
  chmod +x /usr/local/bin/mobie && \
  /usr/local/bin/mobie && \
  echo Finished installing mobie && \
  date

FileNotFoundError: [Errno 2] No such file or directory: '/home/jovyan/.m2/repository/org/jruby/joni/joni/2.1.29/joni-2.1.29.jar' -> '/opt/conda/envs/mobie-java/jgo/org.embl.mobie/mobie-viewer-fiji/3.0.12/32da8e8da4470d39be2214f164c889f873609a9e59ff5ff0db64668c96b510b9/joni-2.1.29.jar'

tischi avatar Jan 31 '23 16:01 tischi

This is what conda installs for the mobie env:

  + bzip2                1.0.8  h0d85af4_4          conda-forge/osx-64     Cached
  + ca-certificates  2022.12.7  h033912b_0          conda-forge/osx-64     Cached
  + jgo                  1.0.4  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + libffi               3.4.2  h0d85af4_5          conda-forge/osx-64     Cached
  + libsqlite           3.40.0  ha978bb4_0          conda-forge/osx-64     Cached
  + libzlib             1.2.13  hfd90126_4          conda-forge/osx-64     Cached
  + maven                3.8.7  h694c41f_0          conda-forge/osx-64     Cached
  + mobie               3.0.12  pyh1a96a4e_0        conda-forge/noarch        8kB
  + ncurses                6.3  h96cf925_1          conda-forge/osx-64     Cached
  + openjdk            8.0.332  hac89ed1_0          conda-forge/osx-64     Cached
  + openssl              3.0.7  hfd90126_2          conda-forge/osx-64        2MB
  + pip                 22.3.1  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + psutil               5.9.4  py311h5547dcb_0     conda-forge/osx-64     Cached
  + python              3.11.0  he7542f4_1_cpython  conda-forge/osx-64     Cached
  + python_abi            3.11  3_cp311             conda-forge/osx-64     Cached
  + readline             8.1.2  h3899abd_0          conda-forge/osx-64     Cached
  + setuptools          66.1.1  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + tk                  8.6.12  h5dbffcc_0          conda-forge/osx-64     Cached
  + tzdata               2022g  h191b570_0          conda-forge/noarch     Cached
  + wheel               0.38.4  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + xz                   5.2.6  h775f41a_0          conda-forge/osx-64     Cached

There is in fact no "joni"...should it be there?

tischi avatar Jan 31 '23 16:01 tischi

@tischi joni is a JAR file, so a Maven dependency, so it won't appear in the list of conda dependencies. The jgo project makes it so that we don't need to bundle every single Java JAR file as its own conda package, which would be a prohibitively large endeavor.

In fact we were not even sure what the -> means in this context.

Symlink, probably, although the error message is odd in that it seems to be pointing backwards to the link. I.e.: /opt/jgo-cache/org.embl.mobie/mobie-viewer-fiji/3.0.12/32da8e8d.../joni-2.1.29.jar is simply a link to the actual file in the local Maven repository cache at ~/.m2/repository/org/jruby/joni/joni/2.1.29/joni-2.1.29.jar—or is supposed to be, but it seems the file is missing. Maven must not have downloaded it. Or maybe you don't have write access to ~/.m2? Or maybe the Maven repository cache is configured to live somewhere else in this environment?

ctrueden avatar Jan 31 '23 18:01 ctrueden

Thanks @ctrueden ! cc @unode

The following works:

Execute during the build:

WORKDIR /opt
RUN source /opt/conda/etc/profile.d/conda.sh && \
  conda create --yes --name mobie-java -c conda-forge python=3.9 mobie && \
  echo $'#!/usr/bin/env -S bash --noprofile --norc\nsource /opt/conda/etc/profile.d/conda.sh\nconda activate mobie-java\nexport JGO_CACHE_DIR=~/.jgo\nmobie "$@"' > /usr/local/bin/mobie && \
  chmod +x /usr/local/bin/mobie && \
  echo Finished installing mobie && \
  date

Note the export JGO_CACHE_DIR=~/.jgo after activating the conda env in order to point to the actual user home directory, where the user has write access.

Note that usr/local/bin/mobie is not run during the build.

After the build

Now, running /usr/local/bin/mobie after the build, where the user is the actual user and not jovan is working!

Conclusion

I don't know how to fix it but to me the issues seem that (i) the location of ~ is different during the build and after the build and (ii) ~ during the build (i.e. /home/jovan) is not writeable for the user after the build.

Digging more into this does not have very high priority for me as it essentially works now.

tischi avatar Feb 01 '23 08:02 tischi

I think this particular issue has a combination of causes that are quite specific to our setup and not of jgo. Specifically, the fact that we are using jgo as part of a recipe to build a docker container, coupled with particularities of the process that include:

  1. Inability to anticipate the user that will execute the container (a JupyterHub container) at container build time.
  2. /home becoming an NFS remote mount when the container is launched via JupyterHub, causing /home/jovyan to be shadowed by other mounts, effectively disappearing from the filesystem.
  3. /home/user and /usr/local living in different filesystems making hard-linking impossible. This may be why jgo defaults to symbolic links instead.

Point 3 is likely the only actionable point from jgo's point of view.

A related issue is that Maven seems to hardcode ~/.m2/repository and we don't have an ENV variable to override this location like we do for jgo. If a way to override it from jgo's side exists, this could help workaround the other points. For instance, by moving all those locations to somewhere outside the HOME of the user.

unode avatar Feb 01 '23 21:02 unode

Hi there, I am also struggling with this. My error is slightly different.

48.74 FileNotFoundError: [Errno 2] No such file or directory: '/opt/m2/repository/ome/formats-gpl/8.4.0-SNAPSHOT/formats-gpl-8.4.0-SNAPSHOT.jar' -> '/opt/.jgo/ome/formats-gpl/LATEST/72c97254a7b442b455d38becbf04681f55d7c075ee75b3bfe961461d29ca8ef0/formats-gpl-8.4.0-SNAPSHOT.jar'

I think I've set most of the variables properly (?) Any ideas will be appreciated!

And here's the Dockerfile

FROM python:3.10.0-slim

ENV VENV_PATH="/opt"
ENV PATH="${VENV_PATH}/bin:${PATH}"
ENV PIP_NO_CACHE_DIR=1
ENV NUMBA_CACHE_DIR=/tmp
# Do not rely on $HOME for caches (Singularity may remap it)
ENV HOME=/opt
# Force jgo/scyjava to use a fixed in-image local Maven repo
ENV GO_CACHE_DIR=/opt/.jgo
ENV MAVEN_USER_HOME=/opt/m2
# If cjdk is ever used, keep its cache away from $HOME
ENV CJDK_CACHE_DIR=/opt/.cjdk

# System deps
RUN apt-get update && apt-get install -y --no-install-recommends \
    openjdk-17-jre-headless ca-certificates procps && \
    rm -rf /var/lib/apt/lists/*

# Python env
RUN python -m venv "$VENV_PATH" && \
    . "${VENV_PATH}/bin/activate" && \
    pip install -U pip wheel setuptools uv

# Create fixed Maven/JGO repo and make it readable
RUN mkdir -p /opt/m2/repository && chmod -R a+rX /opt/m2

# Python deps (explicitly include bioformats_jar so it’s available to prefetch)
RUN . "${VENV_PATH}/bin/activate" && \
    uv pip install --no-cache-dir ImageTileProcessor==0.1.16 pyarrow==19.0.1 bioformats_jar==2020.5.27

# Pre-warm the Maven/JGO cache by importing bioformats_jar and starting the JVM
COPY setup_maven.py /opt/setup_maven.py
RUN . "${VENV_PATH}/bin/activate" && python /opt/setup_maven.py && rm -f /opt/setup_maven.py

# Optional: keep image tidy
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

and the setup_maven.py file:

import scyjava
import os
import bioformats_jar
import jpype

scyjava.config.set_m2_repo("/opt/m2/repository")
scyjava.config.set_cache_dir("/opt/.jgo")

scyjava.config.endpoints.append("ome:formats-gpl:LATEST")
scyjava.start_jvm()
loci = jpype.JPackage("loci")
loci.common.DebugTools.setRootLevel("ERROR")

BioinfoTongLI avatar Oct 10 '25 13:10 BioinfoTongLI