dathere
ModuleNotFoundError: No module named 'datapusher'
After installing DataPusher-Plus for development following the instructions on: https://github.com/dathere/datapusher-plus#datapusher-database-setup . After successful installation of pip install datapusher-plus within the suggested environment: . /usr/lib/ckan/dpplus_venv/bin/activate.
I get the following error after trying to start DataPusher with /usr/lib/ckan/dpplus_venv/bin/uwsgi --enable-threads -i /etc/ckan/datapusher-plus/uwsgi.ini:
[uWSGI] getting INI configuration from /etc/ckan/datapusher-plus/uwsgi.ini
*** Starting uWSGI 2.0.21 (64bit) on [Tue Feb 14 18:11:44 2023] ***
compiled with version: 9.4.0 on 14 February 2023 21:59:44
os: Linux-5.15.0-60-generic #66~20.04.1-Ubuntu SMP Wed Jan 25 09:41:30 UTC 2023
nodename: ubuntu
machine: x86_64
clock source: unix
detected number of CPU cores: 16
current working directory: /home/user
writing pidfile to /tmp/uwsgi.pid
detected binary path: /usr/lib/ckan/dpplus_venv/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
your processes number limit is 23546
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on 0.0.0.0:8800 fd 4
uwsgi socket 0 bound to TCP address 127.0.0.1:44081 (port auto-assigned) fd 3
Python version: 3.8.10 (default, Nov 14 2022, 12:59:47) [GCC 9.4.0]
PEP 405 virtualenv detected: /usr/lib/ckan/dpplus_venv
Set PythonHome to /usr/lib/ckan/dpplus_venv
Python main interpreter initialized at 0x55f3e6fbde50
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 719200 bytes (702 KB) for 9 cores
*** Operational MODE: preforking+threaded ***
spawned uWSGI master process (pid: 3230)
spawned uWSGI worker 1 (pid: 3231, cores: 3)
spawned uWSGI worker 2 (pid: 3232, cores: 3)
spawned uWSGI worker 3 (pid: 3233, cores: 3)
spawned uWSGI http 1 (pid: 3234)
ModuleNotFoundError: No module named 'datapusher'
ModuleNotFoundError: No module named 'datapusher'
unable to load app 0 (mountpoint='') (callable not found or import error)
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** no app loaded. going in full dynamic mode ***
ModuleNotFoundError: No module named 'datapusher'
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
Under dpplus_venv when trying to import datapusher:
(dpplus_venv) user@ubuntu:~$ python
Python 3.8.10 (default, Nov 14 2022, 12:59:47)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import datapusher
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'datapusher'
I have double checked that pip install datapusher-plus:
(dpplus_venv) user@ubuntu:~$ pip install datapusher-plus
Requirement already satisfied: datapusher-plus in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (0.10.2)
Requirement already satisfied: ckanserviceprovider>=1.0 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from datapusher-plus) (1.1.0)
Requirement already satisfied: uwsgi in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from datapusher-plus) (2.0.21)
Requirement already satisfied: requests in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from datapusher-plus) (2.28.2)
Requirement already satisfied: python-dotenv in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from datapusher-plus) (0.21.1)
Requirement already satisfied: psycopg2-binary in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from datapusher-plus) (2.9.5)
Requirement already satisfied: semver in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from datapusher-plus) (2.13.0)
Requirement already satisfied: datasize in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from datapusher-plus) (1.0.0)
Requirement already satisfied: APScheduler<3.10.0,>=2.1.2 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from ckanserviceprovider>=1.0->datapusher-plus) (3.9.1.post1)
Requirement already satisfied: SQLAlchemy<1.4.0,>=1.3.15 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from ckanserviceprovider>=1.0->datapusher-plus) (1.3.24)
Requirement already satisfied: flask-login==0.6.0 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from ckanserviceprovider>=1.0->datapusher-plus) (0.6.0)
Requirement already satisfied: future in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from ckanserviceprovider>=1.0->datapusher-plus) (0.18.3)
Requirement already satisfied: Werkzeug>=1.0.0 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from ckanserviceprovider>=1.0->datapusher-plus) (2.2.3)
Requirement already satisfied: Flask>=1.1.1 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from ckanserviceprovider>=1.0->datapusher-plus) (2.2.2)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from requests->datapusher-plus) (2022.12.7)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from requests->datapusher-plus) (3.4)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from requests->datapusher-plus) (3.0.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from requests->datapusher-plus) (1.26.14)
Requirement already satisfied: tzlocal!=3.*,>=2.0 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from APScheduler<3.10.0,>=2.1.2->ckanserviceprovider>=1.0->datapusher-plus) (4.2)
Requirement already satisfied: six>=1.4.0 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from APScheduler<3.10.0,>=2.1.2->ckanserviceprovider>=1.0->datapusher-plus) (1.16.0)
Requirement already satisfied: setuptools>=0.7 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from APScheduler<3.10.0,>=2.1.2->ckanserviceprovider>=1.0->datapusher-plus) (44.0.0)
Requirement already satisfied: pytz in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from APScheduler<3.10.0,>=2.1.2->ckanserviceprovider>=1.0->datapusher-plus) (2022.7.1)
Requirement already satisfied: MarkupSafe>=2.1.1 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from Werkzeug>=1.0.0->ckanserviceprovider>=1.0->datapusher-plus) (2.1.2)
Requirement already satisfied: itsdangerous>=2.0 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from Flask>=1.1.1->ckanserviceprovider>=1.0->datapusher-plus) (2.1.2)
Requirement already satisfied: Jinja2>=3.0 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from Flask>=1.1.1->ckanserviceprovider>=1.0->datapusher-plus) (3.1.2)
Requirement already satisfied: importlib-metadata>=3.6.0; python_version < "3.10" in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from Flask>=1.1.1->ckanserviceprovider>=1.0->datapusher-plus) (6.0.0)
Requirement already satisfied: click>=8.0 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from Flask>=1.1.1->ckanserviceprovider>=1.0->datapusher-plus) (8.1.3)
Requirement already satisfied: backports.zoneinfo; python_version < "3.9" in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from tzlocal!=3.*,>=2.0->APScheduler<3.10.0,>=2.1.2->ckanserviceprovider>=1.0->datapusher-plus) (0.2.1)
Requirement already satisfied: pytz-deprecation-shim in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from tzlocal!=3.*,>=2.0->APScheduler<3.10.0,>=2.1.2->ckanserviceprovider>=1.0->datapusher-plus) (0.1.0.post0)
Requirement already satisfied: zipp>=0.5 in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from importlib-metadata>=3.6.0; python_version < "3.10"->Flask>=1.1.1->ckanserviceprovider>=1.0->datapusher-plus) (3.13.0)
Requirement already satisfied: tzdata; python_version >= "3.6" in /usr/lib/ckan/dpplus_venv/lib/python3.8/site-packages (from pytz-deprecation-shim->tzlocal!=3.*,>=2.0->APScheduler<3.10.0,>=2.1.2->ckanserviceprovider>=1.0->datapusher-plus) (2022.7)
I am using Ubuntu 20.04 as recommended:
(dpplus_venv) user@ubuntu:~$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.5 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.5 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
CKAN server on the other hand is running fine. Only DataPusher seems to fail.
Thank you for your help!
Hi @DiegoDVillacreses , I gather you followed the Manual Deployment method for Production Deployment, not the Development Installation procedure. https://github.com/dathere/datapusher-plus#development-installation
Can you do that and let me know if it works.
Also, can you run /usr/lib/ckan/dpplus_venv/bin/pip list and see if datapusher-plus is listed?
You are right U am following the Production Deployment.
This is what I get after /usr/lib/ckan/dpplus_venv/bin/pip list:
user@ubuntu:~$ /usr/lib/ckan/dpplus_venv/bin/pip list
Package Version
--------------------- -----------
APScheduler 3.9.1.post1
backports.zoneinfo 0.2.1
certifi 2022.12.7
charset-normalizer 3.0.1
ckanserviceprovider 1.1.0
click 8.1.3
datapusher-plus 0.10.2
datasize 1.0.0
Flask 2.2.2
Flask-Login 0.6.0
future 0.18.3
idna 3.4
importlib-metadata 6.0.0
itsdangerous 2.1.2
Jinja2 3.1.2
MarkupSafe 2.1.2
pip 20.0.2
pkg-resources 0.0.0
psycopg2-binary 2.9.5
python-dotenv 0.21.1
pytz 2022.7.1
pytz-deprecation-shim 0.1.0.post0
requests 2.28.2
semver 2.13.0
setuptools 44.0.0
six 1.16.0
SQLAlchemy 1.3.24
tzdata 2022.7
tzlocal 4.2
urllib3 1.26.14
uWSGI 2.0.21
Werkzeug 2.2.3
wheel 0.38.4
zipp 3.13.0
Hi @jqnatividad !
Sorry to keep bothering you. Perhaps can you see a potential solution for this issue based on the information I provided or you need more information?
Sorry about the delay @DiegoDVillacreses . We're currently packaging a new release this week and it should make installation easier. Will ping you once it's ready.
This issue still occurs. I use Ubuntu 20.04 and datapusher-plus 0.15.0 as shown in the pip list output. Is it known why this is the case?
Pip list output:
Package Version
APScheduler 3.9.1.post1 blinker 1.6.2 certifi 2023.7.22 charset-normalizer 3.2.0 ckanserviceprovider 1.1.0 click 8.1.7 datapusher-plus 0.15.0 datasize 1.0.0 Flask 2.3.3 Flask-Login 0.6.0 future 0.18.3 idna 3.4 itsdangerous 2.1.2 Jinja2 3.1.2 MarkupSafe 2.1.3 pip 22.0.2 psycopg2-binary 2.9.7 python-dateutil 2.8.2 python-dotenv 1.0.0 pytz 2023.3 requests 2.31.0 semver 3.0.1 setuptools 59.6.0 six 1.16.0 SQLAlchemy 1.3.24 tzdata 2023.3 tzlocal 5.0.1 urllib3 2.0.4 uWSGI 2.0.22 Werkzeug 2.3.7 wheel 0.41.2
