recipes icon indicating copy to clipboard operation
recipes copied to clipboard

Media files not loading (404) but no configuration warnings; Docker/SWAG

Open 0x199x opened this issue 2 years ago • 24 comments

Issue

Hi. I just got Tandoor set up with Linuxserver/SWAG

Images are uploading fine, I get success messages. The system page reports that media files are serving up fine and I don't get any errors in docker-compose logs, but my images are not accessible, they 404.

The proxy config is default/unchanged from the linuxserver.io template.

Any ideas? Thank you

image

image

image

Tandoor Version

1.1.4

OS Version

Ubuntu 22.04

Setup

Docker / Docker-Compose

Reverse Proxy

SWAG

Other

No response

Environment file

# only set this to true when testing/debugging
# when unset: 1 (true) - dont unset this, just for development
DEBUG=0
SQL_DEBUG=0

# HTTP port to bind to
# TANDOOR_PORT=8080

# hosts the application can run under e.g. recipes.mydomain.com,cooking.mydomain.com,...
ALLOWED_HOSTS=recipes.mydomain.com

# random secret key, use for example `base64 /dev/urandom | head -c50` to generate one
# ---------------------------- REQUIRED -------------------------
SECRET_KEY=xxxxxxxxxxxxxxx
# ---------------------------------------------------------------

# your default timezone See https://timezonedb.com/time-zones for a list of timezones
TIMEZONE=America/Chicago

# add only a database password if you want to run with the default postgres, otherwise change settings accordingly
DB_ENGINE=django.db.backends.postgresql
# DB_OPTIONS= {} # e.g. {"sslmode":"require"} to enable ssl
POSTGRES_HOST=db_recipes
POSTGRES_PORT=5432
POSTGRES_USER=djangouser
# ---------------------------- REQUIRED -------------------------
POSTGRES_PASSWORD=xxxxxxxxxxxxxxxxxx
# ---------------------------------------------------------------
POSTGRES_DB=djangodb

# database connection string, when used overrides other database settings.
# format might vary depending on backend
# DATABASE_URL = engine://username:password@host:port/dbname

# the default value for the user preference 'fractions' (enable/disable fraction support)
# default: disabled=0
FRACTION_PREF_DEFAULT=1

# the default value for the user preference 'comments' (enable/disable commenting system)
# default comments enabled=1
COMMENT_PREF_DEFAULT=0
# Users can set a amount of time after which the shopping list is refreshed when they are in viewing mode
# This is the minimum interval users can set. Setting this to low will allow users to refresh very frequently which
# might cause high load on the server. (Technically they can obviously refresh as often as they want with their own scripts)
SHOPPING_MIN_AUTOSYNC_INTERVAL=5

# Default for user setting sticky navbar
# STICKY_NAV_PREF_DEFAULT=1

# If base URL is something other than just / (you are serving a subfolder in your proxy for instance http://recipe_app/recipes/)
# Be sure to not have a trailing slash: e.g. '/recipes' instead of '/recipes/'
# SCRIPT_NAME=/recipes

# If staticfiles are stored at a different location uncomment and change accordingly, MUST END IN /
# this is not required if you are just using a subfolder
# This can either be a relative path from the applications base path or the url of an external host
# STATIC_URL=/static/

# If mediafiles are stored at a different location uncomment and change accordingly, MUST END IN /
# this is not required if you are just using a subfolder
# This can either be a relative path from the applications base path or the url of an external host
# MEDIA_URL=/media/

# Serve mediafiles directly using gunicorn. Basically everyone recommends not doing this. Please use any of the examples
# provided that include an additional nxginx container to handle media file serving.
# If you know what you are doing turn this back on (1) to serve media files using djangos serve() method.
# when unset: 1 (true) - this is temporary until an appropriate amount of time has passed for everyone to migrate
GUNICORN_MEDIA=0

# S3 Media settings: store mediafiles in s3 or any compatible storage backend (e.g. minio)
# as long as S3_ACCESS_KEY is not set S3 features are disabled
# S3_ACCESS_KEY=
# S3_SECRET_ACCESS_KEY=
# S3_BUCKET_NAME=
# S3_REGION_NAME= # default none, set your region might be required
# S3_QUERYSTRING_AUTH=1 # default true, set to 0 to serve media from a public bucket without signed urls
# S3_QUERYSTRING_EXPIRE=3600 # number of seconds querystring are valid for
# S3_ENDPOINT_URL= # when using a custom endpoint like minio

# Email Settings, see https://docs.djangoproject.com/en/3.2/ref/settings/#email-host
# Required for email confirmation and password reset (automatically activates if host is set)
# EMAIL_HOST=
# EMAIL_PORT=
# EMAIL_HOST_USER=
# EMAIL_HOST_PASSWORD=
# EMAIL_USE_TLS=0
# EMAIL_USE_SSL=0
# email sender address (default 'webmaster@localhost')
# DEFAULT_FROM_EMAIL=
# prefix used for account related emails (default "[Tandoor Recipes] ")
# ACCOUNT_EMAIL_SUBJECT_PREFIX=

# allow authentication via reverse proxy (e.g. authelia), leave off if you dont know what you are doing
# see docs for more information https://vabene1111.github.io/recipes/features/authentication/
# when unset: 0 (false)
REVERSE_PROXY_AUTH=0

# Default settings for spaces, apply per space and can be changed in the admin view
# SPACE_DEFAULT_MAX_RECIPES=0 # 0=unlimited recipes
# SPACE_DEFAULT_MAX_USERS=0 # 0=unlimited users per space
# SPACE_DEFAULT_MAX_FILES=0 # Maximum file storage for space in MB. 0 for unlimited, -1 to disable file upload.
# SPACE_DEFAULT_ALLOW_SHARING=1 # Allow users to share recipes with public links

# allow people to create accounts on your application instance (without an invite link)
# when unset: 0 (false)
ENABLE_SIGNUP=0

# If signup is enabled you might want to add a captcha to it to prevent spam
# HCAPTCHA_SITEKEY=
# HCAPTCHA_SECRET=

# if signup is enabled you might want to provide urls to data protection policies or terms and conditions# TERMS_URL=
# PRIVACY_URL=
# IMPRINT_URL=

# enable serving of prometheus metrics under the /metrics path
# ATTENTION: view is not secured (as per the prometheus default way) so make sure to secure it
# trough your web server (or leave it open of you dont care if the stats are exposed)
# ENABLE_METRICS=0

# allows you to setup OAuth providers
# see docs for more information https://vabene1111.github.io/recipes/features/authentication/
# SOCIAL_PROVIDERS = allauth.socialaccount.providers.github, allauth.socialaccount.providers.nextcloud,

# Should a newly created user from a social provider get assigned to the default space and given permission by default ?
# ATTENTION: This feature might be deprecated in favor of a space join and public viewing system in the future
# default 0 (false), when 1 (true) users will be assigned space and group
# SOCIAL_DEFAULT_ACCESS = 1

# if SOCIAL_DEFAULT_ACCESS is used, which group should be added
# SOCIAL_DEFAULT_GROUP=guest

# Django session cookie settings. Can be changed to allow a single django application to authenticate several applications
# when running under the same database
# SESSION_COOKIE_DOMAIN=.example.com
# SESSION_COOKIE_NAME=sessionid # use this only to not interfere with non unified django applications under the same top level domain

# by default SORT_TREE_BY_NAME is disabled this will store all Keywords and Food in the order they are created
# enabling this setting makes saving new keywords and foods very slow, which doesn't matter in most usecases.
# however, when doing large imports of recipes that will create new objects, can increase total run time by 10-15x
# Keywords and Food can be manually sorted by name in Admin
# This value can also be temporarily changed in Admin, it will revert the next time the application is started
# This will be fixed/changed in the future by changing the implementation or finding a better workaround for sorting
# SORT_TREE_BY_NAME=0
# LDAP authentication
# default 0 (false), when 1 (true) list of allowed users will be fetched from LDAP server
#LDAP_AUTH=
#AUTH_LDAP_SERVER_URI=
#AUTH_LDAP_BIND_DN=
#AUTH_LDAP_BIND_PASSWORD=
#AUTH_LDAP_USER_SEARCH_BASE_DN=
#AUTH_LDAP_TLS_CACERTFILE=

# Enables exporting PDF (see export docs)
# Disabled by default, uncomment to enable
ENABLE_PDF_EXPORT=1

# Recipe exports are cached for a certain time by default, adjust time if needed
# EXPORT_FILE_CACHE_DURATION=600

Docker-Compose file

---
version: "2.1"
services: 
  db_recipes:
    restart: always
    container_name: db_recipes
    image: postgres:11-alpine
    volumes:
      - /home/lsio/recipes/db:/var/lib/postgresql/data
    env_file:
      - /root/compose/recipes/.env

  recipes:
    image: vabene1111/recipes
    container_name: recipes
    restart: unless-stopped
    env_file:
      - /root/compose/recipes/.env
    environment:
      - UID=1001
      - GID=1001
      - TZ=America/Chicago
    volumes:
      - /home/lsio/recipes/static:/opt/recipes/staticfiles
      - /home/lsio/recipes/media:/opt/recipes/mediafiles
    depends_on:
      - db_recipes
networks:
  default:
    external:
      name: lsio

Relevant logs

No errors

0x199x avatar Apr 24 '22 16:04 0x199x

It appears you aren’t using nginx or other reverse proxy. If that is true gunicorn must be enabled.

GUNICORN_MEDIA=1

smilerz avatar Apr 24 '22 17:04 smilerz

It appears you aren’t using nginx or other reverse proxy. If that is true gunicorn must be enabled.

GUNICORN_MEDIA=1

Yep it's behind nginx. I'm using SWAG (nginx/letsencrypt) .. also mentioned in the original issue that I'm using the default proxy config (the one provided by SWAG) unmodified from its original template. (/config/nginx/proxy-configs/recipes.subdomain.conf)

I'm not using any external mountable storage for the media, it's just placed in my /home/lsio/recipes/ directory under a /media/ subdirectory as defined by my docker-compose. The uploaded files appear there correctly, but they 404 in Tandoor.

$ ls /home/lsio/recipes/media/recipes/
795f2e16-19ff-45e2-8dc4-ca04b62ccf6c_1.jpeg

nginx:

## Version 2021/05/18
# make sure that your dns has a cname set for recipes
# make sure to mount /media/ in your swag container to point to your Recipes Media directory

# if using Authelia use this one:
# Doc: https://vabene1111.github.io/recipes/install/docker/#using-proxy-authentication

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name recipes.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;

    # enable for ldap auth, fill in ldap details in ldap.conf
    #include /config/nginx/ldap.conf;

    # enable for Authelia
    #include /config/nginx/authelia-server.conf;

    # serve media files
    location /media/ {
        alias /media/;
    }

    location / {
        # enable the next two lines for http auth
        #auth_basic "Restricted";
        #auth_basic_user_file /config/nginx/.htpasswd;

        # enable the next two lines for ldap auth
        #auth_request /auth;
        #error_page 401 =200 /ldaplogin;

        # enable for Authelia
        #include /config/nginx/authelia-location.conf;

        include /config/nginx/proxy.conf;
        include /config/nginx/resolver.conf;
        set $upstream_app recipes;
        set $upstream_port 8080;
        set $upstream_proto http;
        proxy_pass $upstream_proto://$upstream_app:$upstream_port;

    }
}

ghost avatar Apr 24 '22 17:04 ghost

Ah got it.

You are serving up /media but your pictures are located at /home/lsio/recipes/media

smilerz avatar Apr 24 '22 18:04 smilerz

I tried that as well :\

404 (tested with and without trailing slash)

    location /media/ {
        alias /home/lsio/recipes/media/;
    }

I also tried updating the MEDIA_URL env var to MEDIA_URL=/home/lsio/recipes/media/ in combination with the above nginx config.

404

    location /media/ {
        alias /media/;
    }

I wasn't entirely sure if the /media/ path was relative to the bind mounts in the compose or if I had to enter the full path.. but neither seems to be working

ghost avatar Apr 24 '22 18:04 ghost

Are there pictures in that directory?

smilerz avatar Apr 24 '22 18:04 smilerz

Yep.. it's there

ls /home/lsio/recipes/media/recipes/
795f2e16-19ff-45e2-8dc4-ca04b62ccf6c_1.jpeg

image

ghost avatar Apr 24 '22 18:04 ghost

Once you configure the alias correctly please share the relevant logs.

smilerz avatar Apr 24 '22 18:04 smilerz

Here are some fresh logs.

Here are the steps included in this log:

  1. Restarted container
  2. Uploaded a picture
  3. Refreshed the page
$ docker-compose logs
Attaching to recipes, db_recipes
recipes       | Checking configuration...
recipes       | Waiting for database to be ready...
recipes       | Database is ready
recipes       | Migrating database
recipes       | Operations to perform:
recipes       |   Apply all migrations: account, admin, auth, authtoken, contenttypes, cookbook, sessions, sites, socialaccount
recipes       | Running migrations:
recipes       |   No migrations to apply.
recipes       |   Your models in app(s): 'cookbook' have changes that are not yet reflected in a migration, and so won't be applied.
recipes       |   Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.
recipes       | Generating static files
recipes       | js-reverse file written to /opt/recipes/cookbook/static/django_js_reverse
recipes       | 
recipes       | 1 static file copied to '/opt/recipes/staticfiles', 1032 unmodified, 1956 post-processed.
recipes       | Done
recipes       | [2022-04-24 13:27:44 -0500] [1] [INFO] Starting gunicorn 20.1.0
recipes       | [2022-04-24 13:27:44 -0500] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
recipes       | [2022-04-24 13:27:44 -0500] [1] [INFO] Using worker: sync
recipes       | [2022-04-24 13:27:44 -0500] [11] [INFO] Booting worker with pid: 11
recipes       | 172.19.0.3 - - [24/Apr/2022:13:28:55 -0500] "PUT /api/recipe/1/image/ HTTP/1.1" 200 70 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes       | 172.19.0.3 - - [24/Apr/2022:13:29:02 -0500] "GET /edit/recipe/internal/1/ HTTP/1.1" 200 17992 "https://recipes.mydomain.com/view/recipe/1" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes       | 172.19.0.3 - - [24/Apr/2022:13:29:02 -0500] "GET /static/django_js_reverse/reverse.4bbfcb16b9d1.js HTTP/1.1" 200 0 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes       | 172.19.0.3 - - [24/Apr/2022:13:29:03 -0500] "GET /api/recipe/1/ HTTP/1.1" 200 623 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes       | 172.19.0.3 - - [24/Apr/2022:13:29:03 -0500] "GET /api/unit/?query=&page=1&page_size=25 HTTP/1.1" 200 52 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes       | 172.19.0.3 - - [24/Apr/2022:13:29:03 -0500] "GET /api/food/?query=&page=1&page_size=25 HTTP/1.1" 200 52 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes       | 172.19.0.3 - - [24/Apr/2022:13:29:03 -0500] "GET /api/keyword/?query=&page=1&page_size=25 HTTP/1.1" 200 52 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes       | 172.19.0.3 - - [24/Apr/2022:13:29:03 -0500] "GET /api/user-file/?query= HTTP/1.1" 200 2 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes       | 172.19.0.3 - - [24/Apr/2022:13:29:03 -0500] "GET /api/recipe/?query=&page=1&page_size=25 HTTP/1.1" 200 554 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes       | 172.19.0.3 - - [24/Apr/2022:13:29:03 -0500] "GET /service-worker.js HTTP/1.1" 200 54271 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"

ghost avatar Apr 24 '22 18:04 ghost

You’ll need to provide web logs from swag. Tandoor isn’t going to receive the image requests.

smilerz avatar Apr 24 '22 18:04 smilerz

swag nginx log

x.x.x.x - - [24/Apr/2022:13:28:55 -0500] "PUT /api/recipe/1/image/ HTTP/2.0" 200 70 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:28:55 -0500] "GET /media/recipes/85a4a9ae-368b-4fbf-8fd0-52fb9191fc49_1.jpeg HTTP/2.0" 404 107 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:02 -0500] "GET /edit/recipe/internal/1/ HTTP/2.0" 200 3604 "https://recipes.mydomain.com/view/recipe/1" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:02 -0500] "GET /static/django_js_reverse/reverse.4bbfcb16b9d1.js HTTP/2.0" 200 4909 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /api/recipe/1/ HTTP/2.0" 200 623 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /api/unit/?query=&page=1&page_size=25 HTTP/2.0" 200 52 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /api/food/?query=&page=1&page_size=25 HTTP/2.0" 200 52 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /api/keyword/?query=&page=1&page_size=25 HTTP/2.0" 200 52 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /api/user-file/?query= HTTP/2.0" 200 2 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /api/recipe/?query=&page=1&page_size=25 HTTP/2.0" 200 554 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /media/recipes/85a4a9ae-368b-4fbf-8fd0-52fb9191fc49_1.jpeg HTTP/2.0" 404 107 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /service-worker.js HTTP/2.0" 200 54271 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"

ghost avatar Apr 24 '22 18:04 ghost

Small suggestion while we're chatting about this (thank you for your help by the way)

On the "System" screen in Tandoor, I think maybe this message should include a check to make sure the media files are accessible

image

ghost avatar Apr 24 '22 18:04 ghost

That log just shows the 404. You’ll need a deeper level of logging or another source of logs to determine why.

I agree that it would be helpful, but tandoor has no visibility into how those media files are accessed, so I’m not sure how that would even be accomplished.

smilerz avatar Apr 24 '22 18:04 smilerz

That log just shows the 404. You’ll need a deeper level of logging or another source of logs to determine why.

I agree that it would be helpful, but tandoor has no visibility into how those media files are accessed, so I’m not sure how that would even be accomplished.

Maybe something like ..

Create a dummy file to /media -> requests.get(https://full.path/to/media/file.jpeg) -> Check response -> Test OK -> remove dummy file

Just spitballing

I will dig more into this and see if I can figure something out. Thank you. I'm sure it's just a simple configuration issue somewhere.

ghost avatar Apr 24 '22 19:04 ghost

Ah ha.. I think I figured out my problem. Now to figure out the best solution.

I have swag/nginx running under UID 1003 I have Tandoor running under UID 1001

Tandoor's media files are under UID 1001's home folder. Nginx (1003) is trying to access 1001's home folder.

So when nginx tries to alias /home/user1001/recipes/media it's lacking the permission to access those files.

Thus.. 404 file not found. I believe it is a permission error.

Up until this point, I haven't had trouble keeping the swag/lsio containers separated into different linux users, but this time it's an issue since I need to expose the media directory in nginx from a directory it has no permission to access.

EDIT: Er... maybe not. I tested this theory by moving the media outside of the home dir into the system /media/ dir. Same issue. Writes work fine for saving media, but still 404's. Ugh. If I had stopped to consider the fact that writing worked before, I wouldn't even bothered testing this. Definitely not a permission problem.

ghost avatar Apr 24 '22 20:04 ghost

So that didn't work.. but I got some better logs.

This has me scratching my head. It says file not found and it's looking for it in the correct path.. but...

2022/04/24 19:03:45 [error] 518#518: *1 open() "/home/lsio/recipes/media/recipes/5a035438-2482-4c18-86dc-b147ab26c67e_1.jpeg" failed (2: No such file or directory), client: x.x.x.x, server: recipes.*, request: "GET /media/recipes/5a035438-2482-4c18-86dc-b147ab26c67e_1.jpeg HTTP/2.0", host: "recipes.mydomain.com", referrer: "https://recipes.mydomain.com/search/"
2022/04/24 19:04:08 [error] 518#518: *1 open() "/home/lsio/recipes/media/recipes/5a035438-2482-4c18-86dc-b147ab26c67e_1.jpeg" failed (2: No such file or directory), client: x.x.x.x, server: recipes.*, request: "GET /media/recipes/5a035438-2482-4c18-86dc-b147ab26c67e_1.jpeg HTTP/2.0", host: "recipes.mydomain.com", referrer: "https://recipes.mydomain.com/search/"
2022/04/24 19:04:47 [warn] 518#518: *31 an upstream response is buffered to a temporary file /var/lib/nginx/tmp/proxy/2/00/0000000002 while reading upstream, client: x.x.x.x, server: recipes.*, request: "GET /static/vue/js/chunk-vendors.bb5a09569e79.js HTTP/2.0", upstream: "http://172.19.0.9:8080/static/vue/js/chunk-vendors.bb5a09569e79.js", host: "recipes.mydomain.com", referrer: "https://recipes.mydomain.com/search/"
2022/04/24 19:04:50 [error] 518#518: *31 open() "/home/lsio/recipes/media/recipes/5a035438-2482-4c18-86dc-b147ab26c67e_1.jpeg" failed (2: No such file or directory), client: x.x.x.x, server: recipes.*, request: "GET /media/recipes/5a035438-2482-4c18-86dc-b147ab26c67e_1.jpeg HTTP/2.0", host: "recipes.mydomain.com", referrer: "https://recipes.mydomain.com/search/"

The file that it says is not found, is clearly there....

# ls -la /home/lsio/recipes/media/recipes/
total 228
drwxr-xr-x 2 root root   4096 Apr 25 00:03 .
drwxr-xr-x 3 root root   4096 Apr 25 00:03 ..
-rw-r--r-- 1 root root 223091 Apr 25 00:03 5a035438-2482-4c18-86dc-b147ab26c67e_1.jpeg
root@server:~/compose/recipes# 

ghost avatar Apr 25 '22 00:04 ghost

What user is running swag? The file is owned by root.

smilerz avatar Apr 25 '22 00:04 smilerz

swag is running swag, lsio is the UID on my compose (/home/lsio is where the files go).

I've already tried recursively chowning the /media/ path to both the lsio and swag users.. sadly there's no effect

ghost avatar Apr 25 '22 00:04 ghost

Here is a put and a get side by side.. same file uploaded and then attempted to be downloaded

2022/04/24 19:37:02 [warn] 517#517: *1 a client request body is buffered to a temporary file /var/lib/nginx/tmp/client_body/0000000002, client: x.x.x.x, server: recipes.*, request: "PUT /api/recipe/1/image/ HTTP/2.0", host: "recipes.mydomain.com", referrer: "https://recipes.mydomain.com/edit/recipe/internal/1/"

2022/04/24 19:37:03 [error] 517#517: *1 open() "/home/lsio/recipes/media/recipes/8cb2467b-7bf4-4afa-96d7-9c1fe26167ec_1.jpeg" failed (2: No such file or directory), client: x.x.x.x, server: recipes.*, request: "GET /media/recipes/8cb2467b-7bf4-4afa-96d7-9c1fe26167ec_1.jpeg HTTP/2.0", host: "recipes.mydomain.com", referrer: "https://recipes.mydomain.com/edit/recipe/internal/1/"

ghost avatar Apr 25 '22 00:04 ghost

Put is to tandoor, get is from nginx. Not sure that is going to help much. It appears tandoor is accessing the file system as root, which nginx doesn’t won’t have access to. I don’t know enough about docker to help change that behavior.

smilerz avatar Apr 25 '22 01:04 smilerz

Thanks, I'll keep digging.

I may have possibly hit on something. I changed the /media/ path in my nginx proxy config to media/ (no leading slash) and it spit out this

514#514: *1 open() "/var/lib/nginx/media/recipes/8cb2467b-7bf4-4afa-96d7-9c1fe26167ec_1.jpeg" failed (2: No such file or directory)

.. which is internal to the container. So I'm wondering if that's why I'm getting a 404 for a file that clearly exists on the filesystem. When it says it's looking for /home/lsio/..../media, it's actually looking at <container_root>/home/lsio/.../media which doesn't exist.

Maybe.. possibly. tableflip.gif

ghost avatar Apr 25 '22 03:04 ghost

Did you get this sorted?

The way I have it working is:

  • Media files are located at "/docker/volumes/recipes/mediafiles"
  • mediafiles folder permissions are set to the same as SWAG user
  • That location is mounted into SWAG with the below compose config
services:
  swag:
    image: lscr.io/linuxserver/swag
    container_name: swag
    ...
    volumes:
      - ./volumes/swag:/config
      - ./volumes/swag/nginx/conf.d:/etc/nginx/conf.d
      - ./volumes/recipes/mediafiles:/recipes/media
    ...
    restart: always

Recipes environment variable "MEDIA_URL"

MEDIA_URL=https://recipes.domain.com/media/

SWAG recipes configuration

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name recipes.*;

    include /config/nginx/ssl.conf;

    include /config/nginx/geoipblock.conf;

    client_max_body_size 0;

    # enable for ldap auth, fill in ldap details in ldap.conf
    #include /config/nginx/ldap.conf;

    # enable for Authelia
    include /config/nginx/authelia-server.conf;
	
    # serve media files
    location /media {
    alias /recipes/media/;
    }
	
    location / {
        # enable the next two lines for http auth
        #auth_basic "Restricted";
        #auth_basic_user_file /config/nginx/.htpasswd;

        # enable the next two lines for ldap auth
        #auth_request /auth;
        #error_page 401 =200 /ldaplogin;

        # enable for Authelia
        #include /config/nginx/authelia-location.conf;

        include /config/nginx/proxy.conf;
        include /config/nginx/resolver.conf;
        set $upstream_app recipes;
        set $upstream_port 8080;
        set $upstream_proto http;
        proxy_pass $upstream_proto://$upstream_app:$upstream_port;

    }

    location ~ /admin|/system {
        # enable the next two lines for http auth
        #auth_basic "Restricted";
        #auth_basic_user_file /config/nginx/.htpasswd;

        # enable the next two lines for ldap auth
        #auth_request /auth;
        #error_page 401 =200 /ldaplogin;

        # enable for Authelia
        include /config/nginx/authelia-location.conf;

        include /config/nginx/proxy.conf;
        resolver 127.0.0.11 valid=30s;
        set $upstream_app recipes;
        set $upstream_port 8080;
        set $upstream_proto http;
        proxy_pass $upstream_proto://$upstream_app:$upstream_port;

    }
}

Hope this helps :)

KibosJ avatar May 01 '22 16:05 KibosJ

I am also getting 404s and I've tried running it multiple ways

this command line doesn't even work for me, cant access it: https://docs.tandoor.dev/install/docker/#docker

plain: https://docs.tandoor.dev/install/docker/#plain

getting 404s without any sort of proxy and with haproxy

jamolnng avatar May 05 '22 05:05 jamolnng

You can try to mount additional volume to swag based on this example

In my case:

version: "3"

networks:
    lsio:
        external:
            name: lsio

services:
  swag:
    image: ghcr.io/linuxserver/swag:latest
    container_name: swag
    cap_add:
      - NET_ADMIN
    environment:
      - PUID=1000
      - PGID=1000
    volumes:
      - ./config:/config
      - ../tandoor/mediafiles:/media
    networks:
      - lsio 
    ports:
      - 443:443
      - 80:80 #optional
    restart: unless-stopped

leangseu avatar May 05 '22 11:05 leangseu

For what it's worth, I managed to get this working by just commenting out this section:

    # serve media files
    location /media/ {
        alias /opt/recipes/mediafiles/;
    }

I have serve gunicorn media set to 0 and I'm hosting on unraid with the lsio swag container.

My nginx config looks like this:

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name cook.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Host $http_host;
    
	
    # serve media files
    #location /media/ {
    #    alias /opt/recipes/mediafiles/;
    #}
	
    location / {
        include /config/nginx/proxy.conf;
        include /config/nginx/resolver.conf;
        set $upstream_app TandoorRecipes;
        set $upstream_port 8080;
        set $upstream_proto http;
        proxy_pass $upstream_proto://$upstream_app:$upstream_port;
    }
}

jkirkcaldy avatar Jun 06 '22 13:06 jkirkcaldy