Nginx & uwsgi

[javawolfpack]

In pull request: https://github.com/UCL-INGI/INGInious/pull/531

@anthonygego mentioned running the inginious-webapp as a uwsgi instance w/ nginx. Can you share your uwsgi configuration for doing that? I can't find any way to run a binary/command in uwsgi. My experience is with Flask/Django which provides a wsgi.py file in their projects.

The inginious-webapp says, "This command can run a standalone web server (see --host and --port options), but also as a FastCGI or WSGI backend." but more details on how that is done could be useful in the documentation.

[anthonygego]

inginious-webapp is a WSGI compatible file. When imported as a module, it provides a WSGI callable application variable. There are two ways to set the WSGI app using uwsgi:

• Using wsgi-file : here you can simply set it to the path to inginious-webapp.
• A more manual way using chdir to set the working directory, module to specify the Python module in which the WSGI callable is found, and callable to set the specific variable referencing to the WSGI app.

Required additional parameters are :

• enable-threads = true because background operations are run.
• lazy-apps = true to tell uwsgi to fork the app earlier in case you want multiple processes running.
• env = INGINIOUS_WEBAPP_CONFIG=<path_to_config_file> to set the configuration file.

Minimal example using a virtualenv:

  [uwsgi]
  socket = /run/uwsgi/inginious.sock
  chmod-socket = 660
  plugins = python38
  enable-threads = true
  lazy-apps = true
  virtualenv = <path_to_my_venv>
  env = INGINIOUS_WEBAPP_CONFIG=<path_to_my_configuration.yaml>
  wsgi-file = <path_to_my_venv>/bin/inginious-webapp

Please note that you may have to rebuild the python38 module to enable uwsgi to run the appropriate Python binary. This may be the case if you are running a Linux distro that predates the release of Python 3.8 (EL7,8 for instance). In this case, you have to install the uwsgi developement package as well and run something like:

PYTHON=<path_to_python38_binary> uwsgi --build-plugin "/usr/src/uwsgi/<version>/plugins/python/ python38" 

and put the produced file into /usr/lib64/uwsgi/. This may vary according to your distro.

The main reason why this is not documented yet is that uwsgi is quite a complex tool we still discover how to tweak after years using it and for which we do not feel confident providing support. I'll keep this issue open so that we do not forget to refactor the docs to add the above information. It would be useful if you could confirm those steps (as well as your Linux distro) in this thread.

[javawolfpack]

That was helpful; my inginious-webapp and inginious-webdav are at least loading now. I can't authenticate into the webdav app for some reason.

I'm on Ubuntu 22.04. Here's my inginious-webapp:

[uwsgi]
env = INGINIOUS_WEBAPP_CONFIG=/var/www/inginious/configuration.yaml
socket = /tmp/inginious-webapp.sock
processes = 2
threads = 2
wsgi-file = /usr/local/bin/inginious-webapp
http-timeout = 86400
http-timeout-as = 86400
plugins = python38
enable-threads = true
lazy-apps = true
vacuum = true
die-on-term = true
master = true
uid = www-data
gid = www-data
chmod-socket = 664
listen = 1024 # set max connections to 1024 in uWSGI

And here's my inginious-webdav:

[uwsgi]
env = INGINIOUS_WEBAPP_CONFIG=/var/www/inginious/configuration.yaml
manage-script-name = true
socket = /tmp/inginious-webdav.sock
processes = 1
threads = 1
wsgi-file = /usr/local/bin/inginious-webdav
http-timeout = 86400
http-timeout-as = 86400
enable-threads = true
lazy-apps = true
vacuum = true
logger = file:/tmp/errlog
die-on-term = true
master = true
uid = www-data
gid = www-data
chmod-socket = 664
listen = 128 # set max connections to 1024 in uWSGI

Not sure if it's a uwsgi issue or config issue regarding webdav, so here's that config currently:

backend: local
backup_directory: /var/www/inginious/backups
local-config:
    debug_host: inginious.csuchico.edu
mongo_opt:
    database: INGInious
    host: localhost
plugins: []
webdav_host: https://inginious.csuchico.edu/webdav
session_parameters:
    ignore_change_ip: false
    secret_key: <removed for security>
    secure: true
    timeout: 86400
superadmins:
- <my account>
tasks_directory: /var/www/inginious/tasks
use_minified_js: true

Not sure if it's because I have the webdav on a different route vs port? Not sure if I can get any port other than 443 open even if also is HTTPS. But if that's the issue can look into that. Thanks.

[anthonygego]

On a different port, without the manage-script-name option, that should work fine.

On a different route that's a bit trickier. Besides, I recall that Windows WebDAV clients send WebDAV requests to the root path. That is, routing inginious-webdav to https://inginious.csuchico.edu/webdav will probably make some clients unhappy. You may be able to filter the requests and route them accordingly but that's probably a lot of headache. In all cases if you want this setup to work, you will need to play with the manage-script-name option, probably with the SCRIPT_NAME variable at nginx side, and see what happens in the debug logs of wsgidav (the integrated libray we use, the debug flag from configuration.yaml should be passed to it)

I would actually recommend you to ask for another subdomain entry, CNAME to your existing one. This way, you don't have to open any additional ports and you remain the most "standard" possible for all the clients. In this case, you don't need the manage-script-name option.

You may be able to test this setup before asking anything by defining two virtual nginx server, one with the server name, that serves inginious-webapp, and the default webserver that serves inginious-webdav. By reaching the server with its IP only, you should be routed to the webdav instance.

[javawolfpack]

Campus IT was pretty fast, so already migrated to a subsubdomain & added a SCRIPT_NAME variable in the uwsgi_params, though made it a blank... is it supposed to have a value? The manage-script-name addition was removed, though was there to prevent an error regarding SCRIPT_NAME not meeting the expected values.

Subsubdomain is routing but still can't login as my login isn't being accepted. Is it supposed to be accessed from the webapp for submissions? Perhaps this is what is leading to the issue in my Q/A question https://github.com/UCL-INGI/INGInious/discussions/948

[anthonygego]

Indeed, my bad, I didn't look through all the config files, you can either set the SCRIPT_NAME to an empty string in this case or let the manage-script-name option do the thing is it is working.

Also ensure that you use the crendetials that are provided in the top of the task list. You need to use the generated API key as password. In case this still does not work, have a look to the logs, and if something weird pops, do not hesistate to copy/paste it here.

[javawolfpack]

Also ensure that you use the crendetials that are provided in the top of the task list. You need to use the generated API key as password.

This was the reason I couldn't login didn't realize there were different credentials generated for it.