Issue with lighttpd/inginious upgrade

[vvandenschrieck]

Hello all,

I've just updated my inginious server to the latest inginious release, and encountered strange issues. I've followed the installation guide from scratch to be sure that my lighttpd configuration is up to date, but my inginious frontend is having problems with URL rewriting for static files :

[Error] Failed to load resource: the server responded with a status of 404 (Not Found) (bootstrap.min.css, line 0) - https://inginious.ephec.be/inginious-webapp/static/css/bootstrap.min.css

It looks like the URL redirection described here is not working, as I see an "inginious-webapp" token inserted in the middle of the static URL.

Any idea about the origin of this issue?

Thanks in advance!

[anthonygego]

lighttpd is known to have behaviour changes even between minor release, but it also can be due to the migration to flask. It would be worth looking at the logs to see where lighttpd actually looks for the file on the disk and understand why it returns a 404 error.

I think I witnessed a similar issue a while ago and the fix was to play with the WSGI SCRIPT_NAME parameter and add a trailing /, but absolutely not sure this is linked. You can probably find more information here: https://flask.palletsprojects.com/en/2.0.x/deploying/fastcgi/#configuring-lighttpd

I do not use lighttpd anymore so it would be nice if the docs could be updated once this issue is fixed.

[vvandenschrieck]

Actually i'm not sure it's an lighttpd issue, as the faulty static URL are generated by the webapp. I'm trying to track down the URL build process, looking into the frontend/app.py get_homepath() that is using request.url_root[:-1]. If you have any insight relative to this function...

[anthonygego]

Indeed, but the URL is generated from the FastCGI parameters that are passed to the app by the webserver. The Flask url_root is computed from the SCRIPT_NAME (tweakable) and PATH_INFO (non-tweakable) sent by the webserver to the application. If the SCRIPT_NAME is incorrectly filled by the webserver, it can result in weird computed paths.

web.py seemed to use REAL_SCRIPT_NAME instead to perform the same kind of tweak so the docs are clearly not valid anymore.

It would be useful to know what paths are computed (they are shown in access error logs) to help you in this issue.

[vvandenschrieck]

Thanks for the explanation, I'm starting to understand :-).

Here is a sample of the access log with the 404 error. I'll have a look on the SCRIPT_NAME variable.

78.46.88.58 inginious.ephec.be - [17/Sep/2021:09:02:14 +0200] "GET / HTTP/1.1" 302 228 "-" "HetrixTools Uptime Monitoring Bot. https://hetrix.tools/uptime-monitoring-bot.html" 78.46.88.58 inginious.ephec.be - [17/Sep/2021:09:02:14 +0200] "GET /courselist HTTP/1.1" 200 12564 "https://inginious.ephec.be/" "HetrixTools Uptime Monitoring Bot. https://hetrix.tools/uptime-monitoring-bot.html" 10.98.0.62 inginious.ephec.be - [17/Sep/2021:09:02:16 +0200] "GET /courselist HTTP/1.1" 301 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15" 10.98.0.62 inginious.ephec.be - [17/Sep/2021:09:02:16 +0200] "GET /courselist HTTP/1.1" 200 12644 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15" 10.98.0.62 inginious.ephec.be - [17/Sep/2021:09:02:16 +0200] "GET /inginious-webapp/static/css/bootstrap.min.css HTTP/1.1" 404 9053 "https://inginious.ephec.be/courselist" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15" 10.98.0.62 inginious.ephec.be - [17/Sep/2021:09:02:16 +0200] "GET /inginious-webapp/static/js/all-minified.js HTTP/1.1" 404 9053 "https://inginious.ephec.be/courselist" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15" 10.98.0.62 inginious.ephec.be - [17/Sep/2021:09:02:16 +0200] "GET /inginious-webapp/static/images/header.png HTTP/1.1" 404 9053 "https://inginious.ephec.be/courselist" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15"

[anthonygego]

Given your logs it's highly probable that lighttpd sets SCRIPT_NAME to /inginious-webapp by default, which is the one specified in the lighttpd config file (fastcgi.server = ( "/inginious-webapp" =>).

You should probably set it to / or leave it empty (this will depends if the PATH_INFOis prepended by a / by lighttpd, because Flask happens to concatenate them some times in the code). Try both and see what it does.

[vvandenschrieck]

I can't find a way to redefine SCRIPT_NAME, looks like the application is overwriting my attemps :-/. I log request.environ in app.py homepath() funtion and always get a script name value of "/inginious-webapp".

I tried by redifining the env variable from command line, in the inginious-webapp script and via the lighttpd.conf, none was effective. How would you do to change it?

[vvandenschrieck]

Ok, I couldn't manage to change the SCRIPT_NAME from outside inginious, but I found a solution involving a middleware in frontend/app.py :

class MiddlewareScriptNameFix():
    def __init__(self, app):
        self.app = app
    def __call__(self, environ, start_response):
        environ['SCRIPT_NAME'] = '/'
        return self.app(environ, start_response)

And in the end of get_app() :

return MiddlewareScriptNameFix(flask_app.wsgi_app), lambda: _close_app(mongo_client, client)

Sorry, that's a quite ugly quickfix, but... it works for now :-).

[gstrauss]

I can't find a way to redefine SCRIPT_NAME

lighttpd creates SCRIPT_NAME based on RFC 3875 "The Common Gateway Interface (CGI) Version 1.1" You can modify lighttpd behavior by telling lighttpd not to check the local filesystem. Try "check-local" => "disable" in lighttpd.conf for the backend config.

https://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_ConfigurationOptions#gw_backend-gateway-server-host-options