datapusher icon indicating copy to clipboard operation
datapusher copied to clipboard

Published 20 hours ago • verified publisher icon ckan

Datapusher fails with error related to _table_metadata

Open abedkhooli opened this issue 5 years ago • 2 comments

I have CKAN 2.8.3 (updated from 2.8.2) on Ubuntu 16.04 and upload to datastore stays pending (push_to_datastore). The suspected error log is below.

[Mon Dec 16 17:36:08.354342 2019] [wsgi:error] [pid 27895:tid 139717909374720] 2019-12-16 17:36:08,354 INFO  [ckan.lib.base]  /dataset/48841c80-71e9-420d-84cd-da4e
d2dd37b9/resource/4d5b837e-ba17-4858-b9e7-c46dbaf5a14e/download/cpi-1996-2018.csv render time 0.018 seconds
[Mon Dec 16 17:36:08.518779 2019] [wsgi:error] [pid 27896:tid 139717900982016] 2019-12-16 17:36:08,516 ERROR [ckan.views.api] (psycopg2.ProgrammingError) relation 
"_table_metadata" does not exist
[Mon Dec 16 17:36:08.518793 2019] [wsgi:error] [pid 27896:tid 139717900982016] LINE 1: SELECT alias_of FROM "_table_metadata"
[Mon Dec 16 17:36:08.518796 2019] [wsgi:error] [pid 27896:tid 139717900982016]                              ^
[Mon Dec 16 17:36:08.518798 2019] [wsgi:error] [pid 27896:tid 139717900982016]  [SQL: 'SELECT alias_of FROM "_table_metadata"\\n                                   
     WHERE name = %(id)s'] [parameters: {'id': u'4d5b837e-ba17-4858-b9e7-c46dbaf5a14e'}]
[Mon Dec 16 17:36:08.518801 2019] [wsgi:error] [pid 27896:tid 139717900982016] Traceback (most recent call last):
[Mon Dec 16 17:36:08.518803 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/src/ckan/ckan/views/api.py", line 288, in action
[Mon Dec 16 17:36:08.518806 2019] [wsgi:error] [pid 27896:tid 139717900982016]     result = function(context, request_data)
[Mon Dec 16 17:36:08.518808 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/src/ckan/ckan/logic/__init__.py", line 464, in wrapped
[Mon Dec 16 17:36:08.518810 2019] [wsgi:error] [pid 27896:tid 139717900982016]     result = _action(context, data_dict, **kw)
[Mon Dec 16 17:36:08.518813 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/src/ckan/ckanext/datastore/logic/action.py", line 451,
 in datastore_search
[Mon Dec 16 17:36:08.518803 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/src/ckan/ckan/views/api.py", line 288, in action
[Mon Dec 16 17:36:08.518806 2019] [wsgi:error] [pid 27896:tid 139717900982016]     result = function(context, request_data)
[Mon Dec 16 17:36:08.518808 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/src/ckan/ckan/logic/__init__.py", line 464, in wrapped
[Mon Dec 16 17:36:08.518810 2019] [wsgi:error] [pid 27896:tid 139717900982016]     result = _action(context, data_dict, **kw)
[Mon Dec 16 17:36:08.518813 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/src/ckan/ckanext/datastore/logic/action.py", line 451,
 in datastore_search
[Mon Dec 16 17:36:08.518815 2019] [wsgi:error] [pid 27896:tid 139717900982016]     res_exists, real_id = backend.resource_id_from_alias(res_id)
[Mon Dec 16 17:36:08.518817 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/src/ckan/ckanext/datastore/backend/postgres.py", line 1843, in resource_id_from_alias
[Mon Dec 16 17:36:08.518820 2019] [wsgi:error] [pid 27896:tid 139717900982016]     results = self._get_read_engine().execute(resources_sql, id=alias)
[Mon Dec 16 17:36:08.518822 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2064, in execute
[Mon Dec 16 17:36:08.518830 2019] [wsgi:error] [pid 27896:tid 139717900982016]     return connection.execute(statement, *multiparams, **params)
[Mon Dec 16 17:36:08.518833 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute
[Mon Dec 16 17:36:08.518835 2019] [wsgi:error] [pid 27896:tid 139717900982016]     return meth(self, multiparams, params)
[Mon Dec 16 17:36:08.518837 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
[Mon Dec 16 17:36:08.518840 2019] [wsgi:error] [pid 27896:tid 139717900982016]     return connection._execute_clauseelement(self, multiparams, params)
[Mon Dec 16 17:36:08.518842 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
[Mon Dec 16 17:36:08.518845 2019] [wsgi:error] [pid 27896:tid 139717900982016]     compiled_sql, distilled_params
[Mon Dec 16 17:36:08.518847 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
[Mon Dec 16 17:36:08.518849 2019] [wsgi:error] [pid 27896:tid 139717900982016]     context)
[Mon Dec 16 17:36:08.518851 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
[Mon Dec 16 17:36:08.518854 2019] [wsgi:error] [pid 27896:tid 139717900982016]     exc_info
[Mon Dec 16 17:36:08.518856 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
[Mon Dec 16 17:36:08.518858 2019] [wsgi:error] [pid 27896:tid 139717900982016]     reraise(type(exception), exception, tb=exc_tb, cause=cause)
[Mon Dec 16 17:36:08.518861 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
[Mon Dec 16 17:36:08.518863 2019] [wsgi:error] [pid 27896:tid 139717900982016]     context)
[Mon Dec 16 17:36:08.518865 2019] [wsgi:error] [pid 27896:tid 139717900982016]   File "/usr/lib/ckan/default/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
[Mon Dec 16 17:36:08.518867 2019] [wsgi:error] [pid 27896:tid 139717900982016]     cursor.execute(statement, parameters)
[Mon Dec 16 17:36:08.518870 2019] [wsgi:error] [pid 27896:tid 139717900982016] ProgrammingError: (psycopg2.ProgrammingError) relation "_table_metadata" does not exist
[Mon Dec 16 17:36:08.518872 2019] [wsgi:error] [pid 27896:tid 139717900982016] LINE 1: SELECT alias_of FROM "_table_metadata"
[Mon Dec 16 17:36:08.518874 2019] [wsgi:error] [pid 27896:tid 139717900982016]                              ^
[Mon Dec 16 17:36:08.518876 2019] [wsgi:error] [pid 27896:tid 139717900982016]  [SQL: 'SELECT alias_of FROM "_table_metadata"\\n

abedkhooli avatar Dec 16 '19 16:12 abedkhooli

Hi,

most likely you have not run permission script https://docs.ckan.org/en/latest/maintaining/datastore.html#set-permissions as the script creates _table_metadata view.

Zharktas avatar Dec 16 '19 16:12 Zharktas

Re-ran postgres=# sudo ckan datastore set-permissions | sudo -u postgres psql --set ON_ERROR_STOP=1 and it executed silently. Does that mean it failed? Also, if I curl -X GET "http://127.0.0.1:5000/api/3/action/datastore_search?resource_id=_table_metadata" I get curl: (7) Failed to connect to 127.0.0.1 port 5000: Connection refused

abedkhooli avatar Dec 16 '19 16:12 abedkhooli