unit
unit copied to clipboard
nginx
ModuleNotFoundError: No module named 'encodings'
I keep getting: ModuleNotFoundError: No module named 'encodings' in the unit error logs when trying to add new configuration.
Fatal Python error: Py_Initialize: Unable to get the locale encoding ModuleNotFoundError: No module named 'encodings'
2020/12/31 19:51:50 [notice] 13294#13294 process 13297 exited with code 0
2020/12/31 19:51:50 [info] 15500#15500 discovery started
2020/12/31 19:51:50 [notice] 15500#15500 module: java 11.0.8 "/usr/lib/unit/modules/java11.unit.so"
2020/12/31 19:51:50 [notice] 15500#15500 module: perl 5.26.1 "/usr/lib/unit/modules/perl.unit.so"
2020/12/31 19:51:50 [notice] 15500#15500 module: php 7.2.24-0ubuntu0.18.04.6 "/usr/lib/unit/modules/php.unit.so"
2020/12/31 19:51:50 [notice] 15500#15500 module: python 2.7.17 "/usr/lib/unit/modules/python2.7.unit.so"
2020/12/31 19:51:50 [notice] 15500#15500 module: python 3.6.9 "/usr/lib/unit/modules/python3.6.unit.so"
2020/12/31 19:51:50 [notice] 15500#15500 module: python 3.7.5 "/usr/lib/unit/modules/python3.7.unit.so"
2020/12/31 19:51:50 [notice] 15500#15500 module: ruby 2.5.1 "/usr/lib/unit/modules/ruby.unit.so"
2020/12/31 19:51:50 [info] 15499#15499 controller started
2020/12/31 19:51:50 [notice] 15499#15499 process 15500 exited with code 0
2020/12/31 19:51:50 [info] 15514#15514 router started
2020/12/31 19:51:50 [info] 15514#15514 OpenSSL 1.1.1 11 Sep 2018, 1010100f
2020/12/31 19:52:40 [info] 15597#15597 "example_python" application started
2020/12/31 19:52:40 [info] 15602#15602 "example_python" application started
2020/12/31 19:54:20 [info] 15766#15766 "example_python" application started
2020/12/31 19:54:20 [info] 15767#15767 "example_python" application started
2020/12/31 19:54:20 [notice] 15499#15499 process 15597 exited with code 0
2020/12/31 19:54:20 [notice] 15499#15499 process 15602 exited with code 0
2020/12/31 19:57:49 [info] 16117#16117 "django" application started
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00007f64d9a9ad80 (most recent call first):
2020/12/31 19:57:49 [alert] 15499#15499 process 16117 exited on signal 6 (core dumped)
2020/12/31 19:57:49 [warn] 15514#15514 failed to start application "django"
2020/12/31 19:57:49 [alert] 15514#15514 failed to apply new conf
sudo curl -X PUT --data-binary @unit.config --unix-socket /var/run/control.unit.sock http://localhost/config unit.config
"applications": {
"django": {
"type": "python 3.7",
"user": "deepak",
"group": "deepak",
"path": "/home/deepak/PycharmProjects/project/venv",
"home": "/home/deepak/PycharmProjects/project",
"module": "wsgi",
"environment": {
"DJANGO_SETTINGS_MODULE": "project.settings",
"DB_ENGINE": "django.db.backends.postgresql",
"DB_NAME": "project",
"DB_HOST": "127.0.0.1",
"DB_PORT": "5434"
}
}
},
"listeners": {
"*:8400": {
"pass": "applications/django"
}
}
}
Hello,
Please check your venv has modules for python3.7. Most likely it was created for python3.6:
$ ls venv/lib/
python3.6
I also hav the same issue and I do not understand how to run unit with django. Python 3.9. unit 1.25.0
$ source venv/bin/activate
(venv) $ pip list --local
Package Version
--------------- ---------
asgiref 3.4.1
certifi 2021.10.8
chardet 4.0.0
Django 3.2.5
django-ckeditor 6.1.0
django-js-asset 1.2.2
gunicorn 20.1.0
idna 2.10
pip 21.3.1
python-dateutil 2.8.2
pytz 2021.1
requests 2.25.1
setuptools 56.0.0
six 1.16.0
sqlparse 0.4.2
urllib3 1.26.6
(venv)
config:
"applications": {
"frontend": {
"type": "python 3.9",
"path": "/home/frontend/altrepo_front/",
"home": "/home/frontend/altrepo_front/venv",
"module": "altrepo_front.wsgi",
"environment": {
"DJANGO_SETTINGS_MODULE": "altrepo_front.settings",
"DB_ENGINE": "django.db.backends.sqlite3"
}
}
}
log:
2021/10/27 10:10:40 [info] 10536#10536 "frontend" application started
Python path configuration:
PYTHONHOME = '/home/frontend/altrepo_front/venv'
PYTHONPATH = (not set)
program name = 'python3'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/usr/bin/python3'
sys.base_prefix = '/home/frontend/altrepo_front/venv'
sys.base_exec_prefix = '/home/frontend/altrepo_front/venv'
sys.platlibdir = 'lib64'
sys.executable = '/usr/bin/python3'
sys.prefix = '/home/frontend/altrepo_front/venv'
sys.exec_prefix = '/home/frontend/altrepo_front/venv'
sys.path = [
'/home/frontend/altrepo_front/venv/lib64/python39.zip',
'/home/frontend/altrepo_front/venv/lib64/python3.9',
'/home/frontend/altrepo_front/venv/lib64/python3.9/lib-dynload',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00007f2ea7b0f380 (most recent call first):
<no Python frame>
2021/10/27 10:10:40 [notice] 6923#6923 process 10536 exited with code 1
2021/10/27 10:10:40 [warn] 6927#6927 failed to start application "frontend"
2021/10/27 10:10:40 [alert] 6927#6927 failed to apply new conf
oops. it's my mistake. There was no access from unit process to the path of venv.conf
But the log is very unformative and it is not clear what you need .
@RiderALT could you please share the details. It is very common error and would be good to let other people to know more about the reason and the way how to avoid it.
this code in src/python/nxt_python.c:nxt_python_start() check access to venv.conf if there is no access, then pep405 mode is not enabled.
nxt_memcpy(path, c->home, len);
nxt_memcpy(path + len, pyvenv, sizeof(pyvenv));
pep405 = (access(path, R_OK) == 0);
nxt_free(path);
next,
if (pep405) {
mbstowcs(nxt_py_home, c->home, len);
mbstowcs(nxt_py_home + len, bin_python, sizeof(bin_python));
Py_SetProgramName(nxt_py_home);
} else {
mbstowcs(nxt_py_home, c->home, len + 1);
Py_SetPythonHome(nxt_py_home);
}
if pep405 is false, unit exports PYTHONHOME environment variable and python3.9 fall with the 'encodings' error.
If you add an entry to the log that there is no access to venv.conf file in home path and pep405 is not activated - it would be much easier and faster to find out the causes of fall.
@RiderALT, can You please explain more detailed according to described situation in post? What do You need to do with PYTHONHOME to avoid Unit error? Maybe something else? I'm facing the same problem, but can do nothing with it unfortunately even having solution :(
@comradeFreeman use gdb to find. I am moved from unit to another solution for a reasons not related to errors in unit.
@comradeFreeman No need to set PYTHONHOME environment variable, instead setup virtual environment path via home option in your application configuration.
@RiderALT If your reasons are related to some functionality missing in Unit or something is better in some other product, then it would be nice if you can share these reasons. Thanks.
@VBart configuration scheme without files in /etc and not obvious rpm packaging scheme for applications - first reason. #501 - another one. And really, i spent a ton of time to search for the reason not to start the Python application - logs are not very informative in my case with python.
But unit is built for ALT Linux https://packages.altlinux.org/en/sisyphus/srpms/unit/ and I look at its development.
Do you have any updates on that issue ? I am facing the exact same problem
@vgiannokostas Please share the details of your environment:
- Exact OS, Unit version, installation method
- How to you start Unit?
- If you start it manually, please share the
envoutput in same shell session.
- If you start it manually, please share the
- Application configuration.
- Exact error message(s),
unit.logcontent
In short, you need to verify the Python version configured for application and used in virtual environment (should be the same), check the presence and access right for pyvenv.cfg file at the root of virtual environment, avoid Python-related environment variables when starting Unit.
How i fixed it on Manjaro:
I edited my systemd unit.service script and manually specified --user and --group parameters since I couldn't get it working with the default values specified by unitd -h (in my case it asked for 'nobody' as user and user's primary group as group).
Then I did the correct chown -R user:group . in my app folder
@mar0x Thank you for your reply. I hope my answers below can help you understand more
- Exact OS
Distributor ID: Ubuntu
Description: Ubuntu 20.04 LTS
Release: 20.04
Codename: focal
- Unit version
unit version: 1.26.1
configured as ./configure --prefix=/usr --state=/var/lib/unit --control=unix:/var/run/control.unit.sock --pid=/var/run/unit.pid --log=/var/log/unit.log
--tmp=/var/tmp --user=unit --group=unit --tests --openssl --modules=/usr/lib/unit/modules --libdir=/usr/lib/x86_64-linux-gnu --cc-opt='-g -O2
-fdebug-prefix-map=/data/builder/debuild/unit-1.26.1/pkg/deb/debuild/unit-1.26.1=. -specs=/usr/share/dpkg/no-pie-compile.specs
-fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --ld-opt='-Wl,-Bsymbolic-functions
-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'
-
Unit installation method I followed these instructions here
-
I start unit like:
sudo systemctl restart unit
- ENV
SHELL=/bin/bash
PWD=/home/ubuntu/mainly-ui/venv
LOGNAME=ubuntu
XDG_SESSION_TYPE=tty
MOTD_SHOWN=pam
HOME=/home/ubuntu
LANG=C.UTF-8
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
VIRTUAL_ENV=/home/ubuntu/mainly-ui/venv
LC_TERMINAL=iTerm2
SSH_CONNECTION=85.24.210.114 52919 172.26.5.212 22
LESSCLOSE=/usr/bin/lesspipe %s %s
XDG_SESSION_CLASS=user
TERM=xterm-256color
LESSOPEN=| /usr/bin/lesspipe %s
USER=ubuntu
LC_TERMINAL_VERSION=3.4.15
SHLVL=1
XDG_SESSION_ID=1062
LC_CTYPE=C.UTF-8
XDG_RUNTIME_DIR=/run/user/1000
PS1=(venv) \[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$
SSH_CLIENT=85.24.210.114 52919 22
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
PATH=/home/ubuntu/mainly-ui/venv/bin:/home/ubuntu/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
SSH_TTY=/dev/pts/0
OLDPWD=/home/ubuntu/mainly-ui
_=/usr/bin/printenv
- Application configuration:
{
"listeners": {
"127.0.0.1:80": {
"pass": "routes"
},
"*:443": {
"pass": "routes",
"tls": {
"certificate": "certbot2"
}
}
},
"routes": [ {
"match": {
"uri": [
"/api/*",
"/docs",
"/docs/*",
"/openapi.json"
]
},
"action": {
"pass": "applications/miranda_webservices"
}
},
{
"match": {
"uri" : "*"
},
"action": {
"share" : "/home/ubuntu/mainly-ui/frontend/out/$uri",
"fallback": {
"share": "/home/ubuntu/mainly-ui/frontend/out/index.html"
}
}
}
],
"applications": {
"miranda_webservices": {
"type": "python",
"path": "/home/ubuntu/mainly-ui/webservice/",
"home": "/home/ubuntu/mainly-ui/venv",
"module": "main",
"protocol": "asgi",
"callable": "app",
"environment": {
"WEBSERVICE_CONFIG_FILE": "/home/ubuntu/webservice_config.yml",
"MIRANDA_LOGFILE": "/home/ubuntu/miranda_log.txt"
}
}
},
"access_log": "/var/log/access.log"
}
- Error message (from /var/log/unit.log)
2022/03/03 14:27:02 [info] 176019#176019 "miranda_webservices" prototype started
2022/03/03 14:27:02 [info] 176020#176020 "miranda_webservices" application started
Python path configuration:
PYTHONHOME = '/home/ubuntu/mainly-ui/venv'
PYTHONPATH = (not set)
program name = 'python3'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/usr/bin/python3'
sys.base_prefix = '/home/ubuntu/mainly-ui/venv'
sys.base_exec_prefix = '/home/ubuntu/mainly-ui/venv'
sys.executable = '/usr/bin/python3'
sys.prefix = '/home/ubuntu/mainly-ui/venv'
sys.exec_prefix = '/home/ubuntu/mainly-ui/venv'
sys.path = [
'/home/ubuntu/mainly-ui/venv/lib/python38.zip',
'/home/ubuntu/mainly-ui/venv/lib/python3.8',
'/home/ubuntu/mainly-ui/venv/lib/python3.8/lib-dynload',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00007f5ff55df940 (most recent call first):
<no Python frame>
2022/03/03 14:27:02 [notice] 176019#176019 app process (isolated 176020) exited with code 1
2022/03/03 14:27:02 [warn] 149010#149010 failed to start application "miranda_webservices"
2022/03/03 14:27:02 [alert] 149010#149010 failed to apply new conf
2022/03/03 14:27:02 [notice] 149005#149005 process 176019 exited with code 0
- pyenv.cfg of the virtual environment exists. It's user and group are set to
unit.
@vgiannokostas Thank you for complete setup information.
I suspect it is access rights issue. The application process (which is started as unit user in your case) have no access to /home/ubuntu/mainly-ui/webservice and /home/ubuntu/mainly-ui/venv. To ensure this check the access rights:
$ ls -ld /home/ubuntu /home/ubuntu/mainly-ui /home/ubuntu/mainly-ui/webservice /home/ubuntu/mainly-ui/venv
Consider to add user and group attributes (see the documentation) for miranda_webservices application. This attributes controls the effective user to run the application. No need to change the pyenv.cfg ownership.
@MrGecko Although the issue can be solved the way you describe, application user and group attributes is more convenient and flexible way to configure effective application process credentials.
Hello all,
it seems I have the same issue, I don't get it. I'm working under Alpine3.15, but tested under Ubuntu 21.10 with the same effects and log entries. I try to get the Python Flask demo application up and running. I use Python 3.9.7. I followed the documentation, still get the error message:
{
"error": "Failed to apply new configuration."
}
Logfile entries are:
2022/06/06 09:44:21 [info] 6061#6061 "flask" prototype started
2022/06/06 09:44:21 [info] 6062#6062 "flask" application started
Python path configuration:
PYTHONHOME = '/root/tmp/venv/'
PYTHONPATH = (not set)
program name = 'python3'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/usr/bin/python3'
sys.base_prefix = '/root/tmp/venv'
sys.base_exec_prefix = '/root/tmp/venv'
sys.platlibdir = 'lib'
sys.executable = '/usr/bin/python3'
sys.prefix = '/root/tmp/venv'
sys.exec_prefix = '/root/tmp/venv'
sys.path = [
'/root/tmp/venv/lib/python39.zip',
'/root/tmp/venv/lib/python3.9',
'/root/tmp/venv/lib/python3.9/lib-dynload',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'
Current thread 0x000055e833d2cd60 (most recent call first):
<no Python frame>
2022/06/06 09:44:21 [notice] 6061#6061 app process (isolated 6062) exited with code 1
2022/06/06 09:44:21 [warn] 5530#5530 failed to start application "flask"
2022/06/06 09:44:21 [alert] 5530#5530 failed to apply new conf
2022/06/06 09:44:21 [notice] 5521#5521 process 6061 exited with code 0
Access rights of the working folder is set to unit:unit
siren:~/tmp# ll
total 8K
drwxr-xr-x 5 unit unit 4.0K Jun 6 09:33 venv
-rw-r--r-- 1 unit unit 110 Jun 6 09:35 wsgi.py
Unit was started as root, subprocesses are of user unit as expected:
siren:/# ps ax | grep unitd
5521 root 0:00 {unitd} unit: main v1.26.1 [/usr/sbin/unitd --no-daemon --log /var/log/unit.log --control unix:/run/control.unit.sock]
5529 unit 0:00 {unitd} unit: controller
5530 unit 0:00 {unitd} unit: router
6457 root 0:00 grep unitd
siren:/#
The configuration is used as shown in the documentation:
siren:~# cat config.json
{
"listeners": {
"*:80": {
"pass": "applications/flask"
}
},
"applications": {
"flask": {
"type": "python 3.9.7",
"path": "/root/tmp/",
"home": "/root/tmp/venv/",
"module": "wsgi",
"callable": "app"
}
}
}
Obviously I missed something - any hints for me, please?
Best Tom
@tom-friedrichsen Try specifying a user in your configuration:
{
"listeners": {
"*:80": {
"pass": "applications/flask"
}
},
"applications": {
"flask": {
"type": "python 3.9.7",
"path": "/root/tmp/",
"home": "/root/tmp/venv/",
"module": "wsgi",
"callable": "app",
"user": "siren",
"group": "siren"
}
}
}
@tom-friedrichsen Try specifying a user in your configuration:
{ "listeners": { "*:80": { "pass": "applications/flask" } },
"applications": { "flask": { "type": "python 3.9.7", "path": "/root/tmp/", "home": "/root/tmp/venv/", "module": "wsgi", "callable": "app", "user": "siren", "group": "siren" } }}
@MagnusOxlund : Thanks a lot for your reply, and pls accept my appology, that I did not answer to my question as soon as I was able to identify ma mistake. Shame on me, it was a stupid beginner mistake:
- I was logged in as root, as I did the whole configuration
- I stored the files in a /root/somesub folder
- I changed access of this somsub folder to unit:unit - but not the whole path starting from "/".
- Noob learning: Never be root, always do everything possible as non root user. Use sudo of required instead.
Best Tom
Try specifying a user in your configuration:
@MagnusOxlund thanks for being an active member of the Unit community! Please contact me and we'll send you some Unit swag: [email protected]
@tom-friedrichsen Not at all 🙂 Thank you for sharing the details, I'm sure that will be valuable to someone else. I'm glad you figured it out.