Ubuntu 20.04

[tvanderbruggen]

[chrisjbillington]

We are testing this, and came across an issue with the web API for uploading instruments. The flask app fails to start:

root@koheron:~# journalctl -u uwsgi.service -f
-- Logs begin at Mon 2022-09-26 09:14:00 CEST. --
Sep 26 09:15:14 koheron uwsgi[1761]:     from ..globals import current_app
Sep 26 09:15:14 koheron uwsgi[1761]:   File "/usr/local/lib/python3.8/dist-packages/flask/globals.py", line 56, in <module>
Sep 26 09:15:14 koheron uwsgi[1761]:     app_ctx: "AppContext" = LocalProxy(  # type: ignore[assignment]
Sep 26 09:15:14 koheron uwsgi[1761]: TypeError: __init__() got an unexpected keyword argument 'unbound_message'
Sep 26 09:15:14 koheron uwsgi[1761]: unable to load app 0 (mountpoint='') (callable not found or import error)
Sep 26 09:15:14 koheron uwsgi[1761]: *** no app loaded. going in full dynamic mode ***
Sep 26 09:15:14 koheron uwsgi[1761]: *** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
Sep 26 09:15:14 koheron uwsgi[1761]: *** uWSGI is running in multiple interpreter mode ***
Sep 26 09:15:14 koheron uwsgi[1761]: spawned uWSGI master process (pid: 1761)
Sep 26 09:15:14 koheron uwsgi[1761]: spawned uWSGI worker 1 (pid: 1762, cores: 1)

This is supposedly due to an old version of werkzeug. I notice the version of werkzeug installed is pinned, but flask is not. Pip (at least the version running on Ubuntu 20.04) is pretty dumb about dependencies, so it probably installed a new flask and an old werkzeug, as requested but too old for the latest flask it installs.

Anyway, after upgrading to werkzeug 2.2.2, there's another problem:

root@koheron:~# journalctl -u uwsgi.service -f
Sep 26 09:14:03 koheron uwsgi[1719]: [2022-09-26 09:14:03,941] ERROR in app: Exception on /api/instruments/details [GET]
Sep 26 09:14:03 koheron uwsgi[1719]: Traceback (most recent call last):
Sep 26 09:14:03 koheron uwsgi[1719]:   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 2525, in wsgi_app
Sep 26 09:14:03 koheron uwsgi[1719]:     response = self.full_dispatch_request()
Sep 26 09:14:03 koheron uwsgi[1719]:   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1822, in full_dispatch_request
Sep 26 09:14:03 koheron uwsgi[1719]:     rv = self.handle_user_exception(e)
Sep 26 09:14:03 koheron uwsgi[1719]:   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request
Sep 26 09:14:03 koheron uwsgi[1719]:     rv = self.dispatch_request()
Sep 26 09:14:03 koheron uwsgi[1719]:   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request
Sep 26 09:14:03 koheron uwsgi[1719]:     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
Sep 26 09:14:03 koheron uwsgi[1719]:   File "/usr/local/api/./app/__init__.py", line 128, in get_instruments_details
Sep 26 09:14:03 koheron uwsgi[1719]:     return jsonify({'instruments': app.instruments_list, 'live_instrument': app.live_instrument })
Sep 26 09:14:03 koheron uwsgi[1719]:   File "/usr/local/lib/python3.8/dist-packages/flask/json/__init__.py", line 342, in jsonify
Sep 26 09:14:03 koheron uwsgi[1719]:     return current_app.json.response(*args, **kwargs)
Sep 26 09:14:03 koheron uwsgi[1719]:   File "/usr/local/lib/python3.8/dist-packages/flask/json/provider.py", line 309, in response
Sep 26 09:14:03 koheron uwsgi[1719]:     f"{self.dumps(obj, **dump_args)}\n", mimetype=mimetype
Sep 26 09:14:03 koheron uwsgi[1719]:   File "/usr/local/lib/python3.8/dist-packages/flask/json/provider.py", line 230, in dumps
Sep 26 09:14:03 koheron uwsgi[1719]:     return json.dumps(obj, **kwargs)
Sep 26 09:14:03 koheron uwsgi[1719]:   File "/usr/lib/python3.8/json/__init__.py", line 234, in dumps
Sep 26 09:14:03 koheron uwsgi[1719]:     return cls(
Sep 26 09:14:03 koheron uwsgi[1719]:   File "/usr/lib/python3.8/json/encoder.py", line 199, in encode
Sep 26 09:14:03 koheron uwsgi[1719]:     chunks = self.iterencode(o, _one_shot=True)
Sep 26 09:14:03 koheron uwsgi[1719]:   File "/usr/lib/python3.8/json/encoder.py", line 257, in iterencode
Sep 26 09:14:03 koheron uwsgi[1719]:     return _iterencode(o, 0)
Sep 26 09:14:03 koheron uwsgi[1719]:   File "/usr/local/lib/python3.8/dist-packages/flask/json/provider.py", line 122, in _default
Sep 26 09:14:03 koheron uwsgi[1719]:     raise TypeError(f"Object of type {type(o).__name__} is not JSON serializable")

This is fixed by decoding the instrument version from a bytestring to a unicode string.

Here's a patch that makes these changes:

diff --git a/os/api/__init__.py b/os/api/__init__.py
index 40366e7..f57c02b 100644
--- a/os/api/__init__.py
+++ b/os/api/__init__.py
@@ -60,7 +60,7 @@ class KoheronApp(Flask):
         version = "0.0.0"

         if (is_file_in_zip(instrument_filename, version_filename)):
-            version = read_file_in_zip(instrument_filename, version_filename)
+            version = read_file_in_zip(instrument_filename, version_filename).decode('utf8')

         instrument["version"] = version
         instrument["is_default"] = is_default
diff --git a/os/scripts/ubuntu-development.sh b/os/scripts/ubuntu-development.sh
index d119823..136833c 100755
--- a/os/scripts/ubuntu-development.sh
+++ b/os/scripts/ubuntu-development.sh
@@ -161,7 +161,7 @@ pip3 install wheel
 pip3 install --upgrade pip==20.2.2
 pip3 install flask
 pip3 install uwsgi
-pip3 install werkzeug==1.0.1
+pip3 install werkzeug==2.2.2
 pip3 install simplejson
 systemctl enable uwsgi
 systemctl enable unzip-default-instrument