unit Flask : ModuleNotFoundError: No module named '*********'

Hello,

I report an error I have (similar to this problem : https://github.com/nginx/unit/issues/96).

I just try to import a personal module in my python file "wsgi_flask.py" with python 3 and I have its error in unit.log : ModuleNotFoundError: No module named 'JEEDOM'. I use python3.7.3. Otherwise, If I try to import this personal module with python 2, it's working !

Details :

config_flask.json (for python 2) : { "listeners": { "*:8282": { "pass": "routes" } },

"routes": [
	{
		"match": {
			"uri": "/test",
			},
            "action": {
                "pass": "applications/flask_app"
            }
	},
],

"applications": {
    "flask_app": {
        "type": "python",
		"home": "/git/eec/APPLICATION/www/flask/venv",
		"path": "/git/eec/APPLICATION/www/flask",
		"module": "wsgi_flask",
		"processes": 1
    }
},

"access_log": "/var/log/access.log"

}

config_flask.json (for python 3) : { "listeners": { "*:8282": { "pass": "routes" } },

"routes": [
	{
		"match": {
			"uri": "/test",
			},
            "action": {
                "pass": "applications/flask_app"
            }
	},
],

"applications": {
    "flask_app": {
        "type": "python3",
		"home": "/git/eec/APPLICATION/www/flask/venv",
		"path": "/git/eec/APPLICATION/www/flask",
		"module": "wsgi_flask",
		"processes": 1
    }
},

"access_log": "/var/log/access.log"

}

wsgi_flask.py :

import json from flask import Flask, jsonify, request from JEEDOM.Class_MySQL_Jeedom import * from Tools.Variables import *

app = Flask(name)

@app.route('/') def test(): return 'TEST'

application = app

unit.log :

2020/11/16 23:20:54 [info] 27019#27019 "flask_app" application started 2020/11/16 23:20:54 [alert] 27019#27019 Python failed to import module "wsgi_flask" Traceback (most recent call last): File "/git/eec/APPLICATION/www/flask/wsgi_flask.py", line 5, in from JEEDOM.Class_MySQL_Jeedom import * ModuleNotFoundError: No module named 'JEEDOM' 2020/11/16 23:20:54 [notice] 14105#14105 process 27019 exited with code 1 2020/11/16 23:20:54 [warn] 14108#14108 failed to start application "flask_app" 2020/11/16 23:20:54 [alert] 14108#14108 failed to apply new conf

Like I said, with python 2 it's working and with python 3 no. Do you have any ideas ?

Cheers

Nov 16 '20 22:11 elder-easy-care

Where is this module placed in your directory structure?

Also, please make sure that Python virtual environment /git/eec/APPLICATION/www/flask/venv is created with the same Python version as you'd like to use in Unit.

Nov 17 '20 10:11 VBart

My module is in /git/eec/JEEDOM.

root@POC-APP-1:/# ll /git/eec/JEEDOM/ drwxr-xr-x 2 eec eec 4.0K Nov 15 11:43 Quotidien drwxr-xr-x 2 eec eec 4.0K Nov 15 11:25 Preparation drwxr-xr-x 4 eec eec 4.0K Nov 15 11:12 . drwxr-xr-x 9 eec eec 4.0K Nov 15 11:12 .. -rw-r--r-- 1 eec eec 0 Nov 15 11:12 init.py -rw-r--r-- 1 eec eec 5.5K Nov 15 11:12 Class_MySQL_Jeedom.py

I have checked, it's the same version of Python for my virtual env and Unit : Python 3.7.3 :

root@POC-APP-1:/# python --version Python 3.7.3

root@POC-APP-1:/git/eec/APPLICATION/www/flask# source venv/bin/activate (venv) root@POC-APP-1:/git/eec/APPLICATION/www/flask# python --version Python 3.7.3

Nov 17 '20 13:11 elder-easy-care

@elder-easy-care Can you try to import this module in your Python environment with the command line interpreter?

# source venv/bin/activate
# python
Python 3.7.3
Type "help", "copyright", "credits" or "license" for more information.
>>> from JEEDOM.Class_MySQL_Jeedom import *

Nov 17 '20 13:11 VBart

Also, this is very strange bacause between your configurations for Python 2 and 3 are no difference. In both cases Unit will load the latest Python 3 version.

Nov 17 '20 13:11 VBart

root@POC-APP-1:/git/eec/APPLICATION/www/flask# source venv/bin/activate
(venv) root@POC-APP-1:/git/eec/APPLICATION/www/flask# python
Python 3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from JEEDOM.Class_MySQL_Jeedom import *
TEST
>>>

Ah OK, so why the difference between python or python3 in "type" in config_flask.json ?

Nov 17 '20 14:11 elder-easy-care

@elder-easy-care This is mysterious. There shouldn't be any difference. By default, if you don't specify any version, it loads the latest one. In order to load Python 2, you have to specify "python 2" in the type field. Are you sure that the difference is only in the "type" field? Could you try diff -u conf1.json conf2.json on both configs?

Nov 17 '20 16:11 VBart

OK so yes it's weird.

root@POC-APP-1:/git/eec/APPLICATION/www/flask# sudo curl -X PUT --data-binary @config_flask_python2.json --unix-socket /var/run/control.unit.sock http://localhost/config/
{
        "success": "Reconfiguration done."
}
root@POC-APP-1:/git/eec/APPLICATION/www/flask# sudo curl -X PUT --data-binary @config_flask_python3.json --unix-socket /var/run/control.unit.sock http://localhost/config/
{
        "error": "Failed to apply new configuration."
}

config_flask_python2.json :

{
    "listeners": {
        "*:8282": {
            "pass": "routes"
        }
    },

    "routes": [
		{
			"match": {
				"uri": "/test",
				},
                "action": {
                    "pass": "applications/flask_app"
                }
		},
	],

    "applications": {
        "flask_app": {
            "type": "python",
			"home": "/git/eec/APPLICATION/www/flask/venv",
			"path": "/git/eec/APPLICATION/www/flask",
			"module": "wsgi_flask",
			"processes": 1
        }
    },

    "access_log": "/var/log/access.log"
}

config_flask_python3.json :

{
    "listeners": {
        "*:8282": {
            "pass": "routes"
        }
    },

    "routes": [
		{
			"match": {
				"uri": "/test",
				},
                "action": {
                    "pass": "applications/flask_app"
                }
		},
	],

    "applications": {
        "flask_app": {
            "type": "python3",
			"home": "/git/eec/APPLICATION/www/flask/venv",
			"path": "/git/eec/APPLICATION/www/flask",
			"module": "wsgi_flask",
			"processes": 1
        }
    },

    "access_log": "/var/log/access.log"
}

I checked again, the only difference is the field "type" .....

Nov 17 '20 21:11 elder-easy-care

root@POC-APP-1:/git/eec/APPLICATION/www/flask# diff -u config_flask_python2.json config_flask_python3.json
--- config_flask_python2.json   2020-11-17 22:37:04.407744842 +0100
+++ config_flask_python3.json   2020-11-17 22:30:23.440656190 +0100
@@ -18,7 +18,7 @@

     "applications": {
         "flask_app": {
-            "type": "python",
+            "type": "python3",
                        "home": "/git/eec/APPLICATION/www/flask/venv",
                        "path": "/git/eec/APPLICATION/www/flask",
                        "module": "wsgi_flask",

Nov 17 '20 21:11 elder-easy-care

@elder-easy-care Please share unit.log file content.

Nov 17 '20 21:11 mar0x

The result for :

root@POC-APP-1:/git/eec/APPLICATION/www/flask# sudo curl -X PUT --data-binary @config_flask_python3.json --unix-socket /var/run/control.unit.sock http://localhost/config/
{
        "error": "Failed to apply new configuration."
}

2020/11/17 22:31:16 [info] 8279#8279 "flask_app" application started
2020/11/17 22:31:16 [alert] 8279#8279 Python failed to import module "wsgi_flask"
Traceback (most recent call last):
  File "/git/eec/APPLICATION/www/flask/wsgi_flask.py", line 5, in <module>
    from JEEDOM.Class_MySQL_Jeedom import *
ModuleNotFoundError: No module named 'JEEDOM'
2020/11/17 22:31:16 [notice] 14105#14105 process 8279 exited with code 1
2020/11/17 22:31:16 [warn] 14108#14108 failed to start application "flask_app"
2020/11/17 22:31:16 [alert] 14108#14108 failed to apply new conf

Nov 17 '20 21:11 elder-easy-care

@elder-easy-care The lines I'm interesting in contains the word module:. It's a list of language modules available to load.

Nov 17 '20 21:11 mar0x

You mean in unit.log or in my wsgi_flask.py ? :

In unit.log you have : ModuleNotFoundError: No module named 'JEEDOM' 2020/11/17 22:31:16 [notice] 14105#14105 process 8279 exited with code 1 2020/11/17 22:31:16 [warn] 14108#14108 failed to start application "flask_app" 2020/11/17 22:31:16 [alert] 14108#14108 failed to apply new conf

wsgi_flask.py

import json from flask import Flask, jsonify, request from JEEDOM.Class_MySQL_Jeedom import * from Tools.Variables import


app = Flask(name)

@app.route('/')
def test():
return 'TEST'

application = app

Nov 17 '20 22:11 elder-easy-care

After Unit starts there should be lines in unit.log like this:

2020/11/18 00:48:48.586 [notice] 60877#1358110 module: python 3.7.6 "/path/to/module/python3.7.unit.so"
2020/11/18 00:48:48.586 [notice] 60877#1358110 module: python 3.8.1 "/path/to/module/python3.8.unit.so"

I'm trying to understand which modules are used. If you don't see it, please restart unitd and try to apply your configs again.

Nov 17 '20 22:11 mar0x

Ah OK you mean that.

2020/11/17 23:30:52 [info] 29769#29769 discovery started
2020/11/17 23:30:52 [notice] 29769#29769 module: php 7.3.19-1~deb10u1 "/usr/lib/unit/modules/php.unit.so"
2020/11/17 23:30:52 [notice] 29769#29769 module: python 2.7.16 "/usr/lib/unit/modules/python2.7.unit.so"
2020/11/17 23:30:52 [notice] 29769#29769 module: python 3.7.3 "/usr/lib/unit/modules/python3.7.unit.so"
2020/11/17 23:30:53 [info] 29766#29766 controller started
2020/11/17 23:30:53 [notice] 29766#29766 process 29769 exited with code 0
2020/11/17 23:30:53 [info] 29774#29774 router started
2020/11/17 23:30:53 [info] 29774#29774 OpenSSL 1.1.1d  10 Sep 2019, 1010104f
2020/11/17 23:30:53 [info] 29775#29775 "flask_app" application started
2020/11/17 23:30:53 [alert] 29775#29775 Python failed to import module "wsgi_flask"
Traceback (most recent call last):
  File "/git/eec/APPLICATION/www/flask/wsgi_flask.py", line 5, in <module>
    from JEEDOM.Class_MySQL_Jeedom import *
ModuleNotFoundError: No module named 'JEEDOM'
2020/11/17 23:30:53 [notice] 29766#29766 process 29775 exited with code 1
2020/11/17 23:30:53 [warn] 29774#29774 failed to start application "flask_app"
2020/11/17 23:30:53 [alert] 29774#29774 failed to apply new conf
2020/11/17 23:30:53 [alert] 29773#29773 failed to apply previous configuration

Nov 17 '20 22:11 elder-easy-care

Could you please replace your wsgi_flask.py with this simple code, that just outputs Python version and test:

import platform

def application(environ, start_response):
    response_body = platform.python_version()
    start_response('200 OK', [])
    return response_body.encode()

Nov 17 '20 22:11 VBart

So I replace my code in wsgi_flask.py by your code. (@VBart)

I tried with "config_flask_python2.json" and "config_flask_python2.json".


root@POC-APP-1:/git/eec/APPLICATION/www/flask# sudo curl -X PUT --data-binary @config_flask_python2.json --unix-socket /var/run/control.unit.sock http://localhost/config/
{
        "success": "Reconfiguration done."
}
root@POC-APP-1:/git/eec/APPLICATION/www/flask# sudo curl -X PUT --data-binary @config_flask_python3.json --unix-socket /var/run/control.unit.sock http://localhost/config/
{
        "success": "Reconfiguration done."
}

Nov 18 '20 07:11 elder-easy-care

To check the source of successfully loaded module please replace your wsgi_flask.py with the following:

import os
import platform
import sys

def application(environ, start_response):
    body = [platform.python_version()]
    try:
        import JEEDOM.Class_MySQL_Jeedom
        body.append(os.path.abspath(JEEDOM.Class_MySQL_Jeedom.__file__))
    except:
        body.append("import JEEDOM.Class_MySQL_Jeedom failed")

    body.extend(sys.path)

    start_response('200 OK', [])
    return "\n".join(body).encode()

Then apply configuration number 1 and get the output of curl 'http://localhost:8282/'. Then apply configuration number 2 and again get the output of curl 'http://localhost:8282/'.

Please share both outputs.

Nov 18 '20 07:11 mar0x

So I replace my code in wsgi_flask.py by your code. (@mar0x)

I tried with "config_flask_python2.json" and "config_flask_python3.json".

root@POC-APP-1:/git/eec/APPLICATION/www/flask# sudo curl -X PUT --data-binary @config_flask_python2.json --unix-socket /var/run/control.unit.sock http://localhost/config/
{
        "success": "Reconfiguration done."
}
root@POC-APP-1:/git/eec/APPLICATION/www/flask# curl http://localhost:8282/
<!DOCTYPE html><title>Error 404</title><p>Error 404.

root@POC-APP-1:/git/eec/APPLICATION/www/flask# sudo curl -X PUT --data-binary @config_flask_python3.json --unix-socket /var/run/control.unit.sock http://localhost/config/
{
        "success": "Reconfiguration done."
}
root@POC-APP-1:/git/eec/APPLICATION/www/flask# curl http://localhost:8282/
<!DOCTYPE html><title>Error 404</title><p>Error 404.

Nov 18 '20 07:11 elder-easy-care

Please repeat the exercise with the correct URL: http://localhost:8282/test:

apply configuration number 1 and get the output of curl 'http://localhost:8282/test'
apply configuration number 2 and again get the output of curl 'http://localhost:8282/test'
share both outputs
Don't panic!

Nov 18 '20 07:11 mar0x

root@POC-APP-1:/git/eec/APPLICATION/www/flask# sudo curl -X PUT --data-binary @config_flask_python2.json --unix-socket /var/run/control.unit.sock http://localhost/config/
{
        "success": "Reconfiguration done."
}
root@POC-APP-1:/git/eec/APPLICATION/www/flask# curl http://localhost:8282/test
3.7.3
import JEEDOM.Class_MySQL_Jeedom failed
/git/eec/APPLICATION/www/flask
/git/eec/APPLICATION/www/flask/venv/lib/python37.zip
/git/eec/APPLICATION/www/flask/venv/lib/python3.7
/git/eec/APPLICATION/www/flask/venv/lib/python3.7/lib-dynload
/usr/lib/python3.7

root@POC-APP-1:/git/eec/APPLICATION/www/flask# sudo curl -X PUT --data-binary @config_flask_python3.json --unix-socket /var/run/control.unit.sock http://localhost/config/
{
        "success": "Reconfiguration done."
}
root@POC-APP-1:/git/eec/APPLICATION/www/flask# curl http://localhost:8282/test
3.7.3
import JEEDOM.Class_MySQL_Jeedom failed
/git/eec/APPLICATION/www/flask
/git/eec/APPLICATION/www/flask/venv/lib/python37.zip
/git/eec/APPLICATION/www/flask/venv/lib/python3.7
/git/eec/APPLICATION/www/flask/venv/lib/python3.7/lib-dynload
/usr/lib/python3.7

Nov 18 '20 08:11 elder-easy-care

Well, both samples works as expected, and as you can see, JEEDOM.Class_MySQL_Jeedom is not found for both cases.

Nov 18 '20 09:11 mar0x

Can you please try again to import this module in your Python environment with the command line interpreter?

# source /git/eec/APPLICATION/www/flask/venv/bin/activate
# python
Python 3.7.3
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import sys
>>> import JEEDOM.Class_MySQL_Jeedom
>>> os.path.abspath(JEEDOM.Class_MySQL_Jeedom.__file__)
>>> sys.path

Nov 18 '20 09:11 mar0x

root@POC-APP-1:/git/eec/APPLICATION/www/flask# source /git/eec/APPLICATION/www/flask/venv/bin/activate
(venv) root@POC-APP-1:/git/eec/APPLICATION/www/flask# python
Python 3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import sys
>>> import JEEDOM.Class_MySQL_Jeedom
TEST
>>> os.path.abspath(JEEDOM.Class_MySQL_Jeedom.__file__)
'/git/eec/JEEDOM/Class_MySQL_Jeedom.py'
>>> sys.path
['', '/git/eec', '/git/eec/APPLICATION/www/flask/venv/lib/python37.zip', '/git/eec/APPLICATION/www/flask/venv/lib/python3.7', '/git/eec/APPLICATION/www/flask/venv/lib/python3.7/lib-dynload', '/usr/lib/python3.7', '/git/eec/APPLICATION/www/flask/venv/lib/python3.7/site-packages']

Just when I use "import JEEDOM.Class_MySQL_Jeedom", we can see the result "TEST" so the module is found in my virtual env.

Nov 18 '20 09:11 elder-easy-care

Now it is clear: the reason is missing directory /git/eec in module search path. Suggest to add PYTHONPATH to your application environment:

{
    "listeners": {
        "*:8282": {
            "pass": "routes"
        }
    },

    "routes": [
		{
			"match": {
				"uri": "/test",
				},
                "action": {
                    "pass": "applications/flask_app"
                }
		},
	],

    "applications": {
        "flask_app": {
            "type": "python",
			"home": "/git/eec/APPLICATION/www/flask/venv",
			"path": "/git/eec/APPLICATION/www/flask",
			"module": "wsgi_flask",
			"processes": 1,
                        "environment": { "PYTHONPATH": "/git/eec" }
        }
    },

    "access_log": "/var/log/access.log"
}

Please try with modified module first, if it shows your module is loaded, then you can change it back to your application.

Nov 18 '20 09:11 mar0x

@elder-easy-care Could you also show the output of printenv PYTHONPATH command? In this case:

# source /git/eec/APPLICATION/www/flask/venv/bin/activate
# printenv PYTHONPATH

and in this:

# deactivate
# printenv PYTHONPATH

?

Nov 18 '20 09:11 VBart

Also please show:

# cat /git/eec/APPLICATION/www/flask/venv/pyvenv.cfg

Nov 18 '20 09:11 VBart

Thanks guys for your helps.

It's working with ""environment": { "PYTHONPATH": "/git/eec" }" in config_flask.son for Python2 and Python3.

So with Python2 the PYTHONPATH was found from the server (I think) and with Python3 wasn't found ....

Nov 18 '20 15:11 elder-easy-care

@elder-easy-care Python2 is NOT used in your configurations. Both configurations are identical and start Python 3.7.3.

Nov 18 '20 17:11 mar0x

If python3 with Apache2 use apt-get install libapache2-mod-wsgi-py3

May 13 '22 10:05 danielgianni

Closed: problem was solved. 👍🏼

Dec 03 '22 11:12 tippexs