/ui/browse route is not accessible from Docker image

[padraic-shafer]

When running the tiled Docker image on my laptop, the "/ui/browse" route returns 404 Not Found ("detail not found" in the JSON returned). There are no problems accessing the root route, nor any problems using the api routes.

Console output...

172.17.0.1:63078 - "GET /?api_key=SECRET HTTP/1.1" 200
172.17.0.1:63080 - "GET /ui/browse?api_key=SECRET HTTP/1.1" 404
172.17.0.1:63082 - "GET /ui/browse/ HTTP/1.1" 404
172.17.0.1:63084 - "GET /ui/browse/?api_key=SECRET HTTP/1.1" 404
172.17.0.1:63086 - "GET /api/node/full// HTTP/1.1" 200

At this point I would not rule out user error. Do others experience this same behavior?

[danielballan]

The UI has its own build step which requires npm to run.

In order to spare our users that complexity, we bundle the pre-built UI in distributions that we publish to PyPI, such that pip install tiled includes the UI, ready to go.

But our Dockerfile installs tiled from the git source, not from PyPI. Therefore, it needs to build the UI. Alternatively, we could distribute Docker images that install from PyPI, instead of or in addition to images built from source. My sense is that---though an image for each commit can be useful for development---images intended for users should generally be based on tagged releases and probably taken from PyPI. Doing that would side-step this issue.

[danielballan]

I'm curious what @dylanmcreynolds thinks.

[danielballan]

And regardless of how we handle the Docker images, we should put a placeholder index.html with a note and a link to some instructions. This would be overwritten when the UI is built.

[dylanmcreynolds]

The docker image already builds the web ui, or at least attempts to. Perhaps this is failing silently? Or, more likely, not going into the right place for fastapi to be picking it up?

[padraic-shafer]

Dylan and I tried a few combinations of docker volume mounts and config.yml data directory, some of which were successful. Here is what worked and what didn't work. Offhand it seems like the data directory should not be in a subdirectory of '/deploy'?

Summary

Successful	Symptomatic
Config in '/app/config'; data in '/data'	Tutorial instructions
Config in '/app/config'; data in '/data2'	Config in '/app/config'; data in '/deploy/example_files'
Config in '/deploy'; data in '/data2'
Config in '/deploy'; data in '/data/example_files'

---

---

Details

Successful configurations

Config in '/app/config'; data in '/data'

docker run -p 8000:8000 -v "$PWD:/app/config" -v "$PWD/example_files:/data" -e TILED_CONFIG=/app/config/config.yml ghcr.io/bluesky/tiled:main

config.yml :

# config.yml
trees:
  - path: /
    tree: tiled.adapters.files:DirectoryAdapter.from_directory
    args:
      directory: "/data"
authentication:
  allow_anonymous_access: true
  single_user_api_key: SECRET

Config in '/app/config'; data in '/data2'

Was there something magic about '/data'? No

docker run -p 8000:8000 -v "$PWD:/app/config" -v "$PWD/example_files:/data2" -e TILED_CONFIG=/app/config/config.yml ghcr.io/bluesky/tiled:main

config.yml :

# config.yml
trees:
  - path: /
    tree: tiled.adapters.files:DirectoryAdapter.from_directory
    args:
      directory: "/data2"
authentication:
  allow_anonymous_access: true
  single_user_api_key: SECRET

Config in '/deploy'; data in '/data2'

Was there something magic about '/app/config'? No

docker run -p 8000:8000 -v "$PWD:/deploy" -v "$PWD/example_files:/data2" -e TILED_CONFIG=/deploy/config.yml ghcr.io/bluesky/tiled:main

config.yml :

# config.yml
trees:
  - path: /
    tree: tiled.adapters.files:DirectoryAdapter.from_directory
    args:
      directory: "/data2"
authentication:
  allow_anonymous_access: true
  single_user_api_key: SECRET

Config in '/deploy'; data in '/data/example_files'

Is it a problem to mount to '/deploy'? No, at least not got the config.yml file

docker run -p 8000:8000 -v "$PWD:/deploy" -v "$PWD:/data" -e TILED_CONFIG=/deploy/config.yml ghcr.io/bluesky/tiled:main

config.yml :

# config.yml
trees:
  - path: /
    tree: tiled.adapters.files:DirectoryAdapter.from_directory
    args:
      directory: "/data/example_files"
authentication:
  allow_anonymous_access: true
  single_user_api_key: SECRET

---

Configurations with reported symptoms

'/api' and '/' can be accessed, but '/ui/browse' is not found.

Tutorial instructions

docker run --rm -p 8000:8000   --mount type=bind,source="$(pwd)",target=/deploy   --env TILED_CONFIG=/deploy/config.yml ghcr.io/bluesky/tiled:main

config.yml :

# config.yml
trees:
  - path: /
    tree: tiled.adapters.files:DirectoryAdapter.from_directory
    args:
      directory: "example_files"
      # directory: "/deploy/example_files"  # This behaves the same as the line above
authentication:
  allow_anonymous_access: true
  single_user_api_key: SECRET

Config in '/app/config'; data in '/deploy/example_files'

Was it a problem using the bind mount notation from the tutorial? No, get the same symptomatic behavior when data directory '$PWD/example_files' is accessed by container at 'deploy/example_files'.

docker run -p 8000:8000 -v "$PWD:/app/config" -v "$PWD:/deploy" -e TILED_CONFIG=/app/config/config.yml ghcr.io/bluesky/tiled:main

config.yml :

# config.yml
trees:
  - path: /
    tree: tiled.adapters.files:DirectoryAdapter.from_directory
    args:
      directory: "/deploy/example_files"
authentication:
  allow_anonymous_access: true
  single_user_api_key: SECRET

[padraic-shafer]

Apologies for the long report. :)

Please let me know if the workaround is acceptable or if you have thoughts for how to better handle this default case.

[danielballan]

Thanks for the report @pshafer-als --- I appreciate you digging into the details here. Dylan and I will give some thought to the best recommendation.

[dylanmcreynolds]

One thing that I notice is that in the bad configuration, the entire deploy directory is being volumume mounted: -v "$PWD:/deploy", but the good case, it is not.

I wonder if there is something in the docker-container's /deploy directory that the tiled frontend app is using?

[dylanmcreynolds]

I think I'm starting to understand, and I had it exactly opposite in the above comment.

In the failing case, the $PWD:/deploy mount installs the full project into the deploy directory. In the succeeding case, the /deploy directory is nearly empty.

The docker file sets the working dir as /deploy here

In the succeeding case, I think the files for the tiled UI are coming from the venv. The dockerfile copies the ui files into there.

When tiled starts up, it creates endpoints for the UI app only if the files are there: https://github.com/bluesky/tiled/blob/5d1f1e07cadde80b924a66b77668b3b54ada04f0/tiled/utils.py#L538

I'm not 100% confident about this analysis, but I think it's on the right track. I think there a very specific foot-gun here related to volume mounting the full tiled source into /deploy. python is then picking up the files there and in that directory, the build UI files don't exist.

[edit: added note about python and the built files]

[dylanmcreynolds]

Would a quick and easy way out of this be to print a warning out of entrypoint.sh if it sees a setup.py? entrypoint.sh will only exist if we're in a tiled docker environment, so we could say something like "Warning: a setup.py file was detected. If you mount the tiled source folder to the /deploy directory, you might not be able to run the Tiled UI."

[danielballan]

I like that. Two possible refinements:

1. Instead of looking for a setup.py check specifically where tiled is being imported from:

import importlib.util
importlib.util.find_spec("tiled").origin

2. Should we warn or exit? As we know, warnings often go unread. Do we want to support users intentionally mounting a source installation? If not, let's just exit with a message.

[dylanmcreynolds]

Yes, @danielballan, that sounds like a better way to detect this situation.

Warn vs Exit: Part of me wants to say warn...it is not unpopular for developers to volume mount their source code in just this way so that they can develop and test in the docker environment, keeping them very close to the deployed environment. Exiting would make this difficult. Not sure how much we need to worry about that. The current setup would present a few additional hurdles for doing that, I think.

[danielballan]

I would be equally happy with "warn" or "fail and tell the dev to set an env var like TILED_MOUNTED_SOURCE or something to suppress the error".

[padraic-shafer]

As a tiled user, I would be ok with either approach.