mod_wsgi
mod_wsgi copied to clipboard
GrahamDumpleton
Python2.7 egg cache ExtractionError: Can't extract file(s) to egg cache when multy-process was started at same time almost with apache2 httpd.conf set WSGIDaemonProcess processes=8 threads=1
Catch EXception:
File "/opt/venv/lib/python2.7/site-packages/pkg_resources.py", line 945, in resource_filename self, resource_name File "/opt/venv/lib/python2.7/site-packages/pkg_resources.py", line 1633, in get_resource_filename self._extract_resource(manager, self._eager_to_zip(name)) File "/opt/venv/lib/python2.7/site-packages/pkg_resources.py", line 1661, in _extract_resource self.egg_name, self._parts(zip_path) File "/opt/venv/lib/python2.7/site-packages/pkg_resources.py", line 1025, in get_cache_path self.extraction_error() File "/opt/venv/lib/python2.7/site-packages/pkg_resources.py", line 991, in extraction_error raise err ExtractionError: Can't extract file(s) to egg cache
The following error occurred while trying to extract file(s) to the Python egg cache: [Errno 17] File exists: '/var/tmp/.python-eggs/xxxxxxxxxx-linux-x86_64.egg-tmp' The Python egg cache directory is currently set to: /var/tmp/.python-eggs
Perhaps your account does not have write access to this directory? You can change the cache directory by setting the PYTHON_EGG_CACHE environment variable to point to an accessible directory.
pkg_resources.py
pkg_resources.py/extraction_error -> get_cache_path/self.extraction_error() -> _bypass_ensure_directory:
def _bypass_ensure_directory(name, mode=0x1FF): # 0777 # Sandbox-bypassing version of ensure_directory() if not WRITE_SUPPORT: raise IOError('"os.mkdir" not supported on this platform.') dirname, filename = split(name) if dirname and filename and not isdir(dirname): _bypass_ensure_directory(dirname) mkdir(dirname, mode)
ExtractionError: use mkdir to create python egg cache dir, when Multi-process in mod_wsgi WSGIDaemonProcess processes=8 threads=1 set, get this error. can you give me some advice？
Read the docs where it talks about Python egg cache issues and set a different path which is unique to the user that your application runs as. As it stands, it looks like a different user has created the directory and thus user your code runs as can't access it.
- https://modwsgi.readthedocs.io/en/master/user-guides/application-issues.html#access-rights-of-apache-user
Read the docs where it talks about Python egg cache issues and set a different path which is unique to the user that your application runs as. As it stands, it looks like a different user has created the directory and thus user your code runs as can't access it.
- https://modwsgi.readthedocs.io/en/master/user-guides/application-issues.html#access-rights-of-apache-user
Add logs
I have also add os.environ in my WSGI application script file and make sure that has same access rights of Apache User, just like:
import os
os.environ['PYTHON_EGG_CACHE'] = '/var/tmp/.python-eggs'
and add log to record launch application script, also catch this exception:
2022-08-03 18:11:22,918 - INFO - [pid:4139708][midd.py][
The following error occurred while trying to extract file(s) to the Python egg cache:
[Errno 17] File exists: '/var/tmp/.python-eggs/xxxxxxxxxxx-linux-x86_64.egg-tmp'
The Python egg cache directory is currently set to:
/var/tmp/.python-eggs
Perhaps your account does not have write access to this directory? You can change the cache directory by setting the PYTHON_EGG_CACHE environment variable to point to an accessible directory.
ps -ef httpd
[root@node1]# ps -ef | grep httpd root 408142 360881 0 18:31 pts/1 00:00:00 grep --color=auto httpd root 4129919 1 0 18:11 ? 00:00:00 /opt/bin/httpd -k start www-data 4139708 4129919 0 18:11 ? 00:00:05 cal-httpd -k start www-data 4139709 4129919 0 18:11 ? 00:00:05 cal-httpd -k start www-data 4139710 4129919 0 18:11 ? 00:00:05 cal-httpd -k start www-data 4139711 4129919 0 18:11 ? 00:00:05 cal-httpd -k start www-data 4139712 4129919 0 18:11 ? 00:00:05 cal-httpd -k start www-data 4139713 4129919 0 18:11 ? 00:00:05 cal-httpd -k start www-data 4139714 4129919 0 18:11 ? 00:00:05 cal-httpd -k start www-data 4139715 4129919 0 18:11 ? 00:00:05 cal-httpd -k start www-data 4139716 4129919 0 18:11 ? 00:00:00 res-httpd -k start www-data 4139717 4129919 0 18:11 ? 00:00:00 res-httpd -k start www-data 4139718 4129919 0 18:11 ? 00:00:00 res-httpd -k start www-data 4139719 4129919 0 18:11 ? 00:00:00 res-httpd -k start www-data 4139720 4129919 0 18:11 ? 00:00:00 res-httpd -k start www-data 4139721 4129919 0 18:11 ? 00:00:00 res-httpd -k start www-data 4139722 4129919 0 18:11 ? 00:00:00 res-httpd -k start www-data 4139723 4129919 0 18:11 ? 00:00:00 res-httpd -k start www-data 4139724 4129919 0 18:11 ? 00:00:00 res-httpd -k start www-data 4139725 4129919 0 18:11 ? 00:00:00 res-httpd -k start www-data 4139726 4129919 0 18:11 ? 00:00:00 res-httpd -k start www-data 4139727 4129919 0 18:11 ? 00:00:00 res-httpd -k start www-data 4139728 4129919 0 18:11 ? 00:00:00 /opt/bin/httpd -k start www-data 4139729 4129919 0 18:11 ? 00:00:00 /opt/bin/httpd -k start www-data 4139730 4129919 0 18:11 ? 00:00:00 /opt/bin/httpd -k start www-data 4140301 4129919 0 18:11 ? 00:00:00 /opt/bin/httpd -k start
set WSGIDaemonProcess processes=8 threads=1 to cal-httpd, but get sub pid 4139708/4139709 was started at 2022-08-03 18:11:22,918 same time. looks like two same time pid mkdir egg cache catch exception. is there any official advice ?
Try something like:
os.environ['PYTHON_EGG_CACHE'] = '/var/tmp/.python-eggs-%s' % os.getpid()
Only problem with this as is will be that the number of directories may keep growing.