ckanext-datapackager
ckanext-datapackager copied to clipboard
frictionlessdata
Extras: There is a schema field with the same name
Overview
Environment: ckan 2.8.2 relevant plugins: spatial, dcat, pages, ...
Unable to import datapackage from examples like:
- https://datahub.io/core/gdp#data
- https://datahub.io/core/covid-19#data
The error I get is always the same so I report it only once:
[Mon Jun 22 10:11:10.052296 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:10,052 DEBUG [ckan.config.middleware.pylons_app] Pylons route match: {'action': u'import_datapackage', 'controller': u'ckanext.datapackager.controllers.datapackage:DataPackageController'} Origin: extension
[Mon Jun 22 10:11:10.052676 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:10,052 DEBUG [ckan.config.middleware] Route support answers for POST /import_datapackage: [(False, 'flask_app'), (True, 'pylons_app', 'extension')]
[Mon Jun 22 10:11:10.052798 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:10,052 DEBUG [ckan.config.middleware] Serving request via pylons_app app
[Mon Jun 22 10:11:10.063865 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:10,063 DEBUG [ckan.logic] check access OK - package_create user=carlo
[Mon Jun 22 10:11:14.506420 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,301 DEBUG [ckan.logic] check access OK - package_create user=carlo
[Mon Jun 22 10:11:14.536244 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,536 DEBUG [ckan.logic.action.create] package_create validate_errs={'extras': [{}, {}, {}, {}, {}, {}, {'key': [u'There is a schema field with the same name']}]} user=carlo package=covid-19 data={'name': u'covid-19', 'title': u'Novel Coronavirus 2019', 'owner_org': u'6d3859b7-0227-45c2-a190-dbc2d2762697', 'private': True, 'state': 'draft', 'extras': [
{'value': u'data-package', 'key': u'profile'},
{'value': u'c3742992a149dcb6bdd66203b3b12abc', 'key': u'hash'},
{'value': '[{"specType": "simple", "spec": {"series": ["Confirmed", "Deaths"], "group": "Date", "type": "line"}, "resources": ["worldwide-aggregated"], "title": "Total world to date"}, {"specType": "simple", "spec": {"series": ["China", "US", "United_Kingdom", "Italy", "France", "Germany", "Spain", "Iran"], "group": "Date", "type": "line"}, "resources": ["key-countries-pivoted"], "title": "Number of confirmed cases in key countries"}, {"specType": "simple", "spec": {"series": ["Mortality rate"], "group": "Date", "type": "bar"}, "resources": [{"name": "worldwide-aggregated", "transform": [{"expressions": ["data[\\'Deaths\\'] / data[\\'Confirmed\\'] * 100 + \\'%\\'"], "type": "formula", "asFields": ["Mortality rate"]}]}], "title": "Mortality rate in percentage"}, {"specType": "simple", "spec": {"series": ["Increase rate"], "group": "Date", "type": "bar"}, "resources": ["worldwide-aggregated"], "title": "Increase rate from previous day in confirmed cases worldwide"}]', 'key': u'views'},
{'value': 638756590, 'key': u'bytes'},
{'value': 1064829, 'key': u'count_of_rows'},
{'value': '{"findability": "published", "created": "2020-03-22T16:03:58.514457", "modified": "2020-06-22T09:20:30.191018", "flowid": "core/covid-19/309", "owner": "core", "ownerid": "core"}', 'key': u'datahub'},
{'value': u'core/covid-19', 'key': u'id'}], 'type': 'dataset'}
[Mon Jun 22 10:11:14.540607 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,540 DEBUG [ckan.logic] check access OK - package_create user=carlo
[Mon Jun 22 10:11:14.542718 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,542 DEBUG [ckan.lib.base] rendering /usr/lib/ckan/default/ckanext-datapackager/ckanext/datapackager/templates/datapackage/import_datapackage.html [jinja2]
[Mon Jun 22 10:11:14.587477 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,587 DEBUG [ckan.logic] check access OK - dashboard_new_activities_count user=carlo
[Mon Jun 22 10:11:14.587948 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,587 DEBUG [ckan.logic] check access OK - dashboard_activity_list user=carlo
[Mon Jun 22 10:11:14.635208 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,635 DEBUG [ckan.logic] check access OK - ckanext_pages_list user=carlo
[Mon Jun 22 10:11:14.635528 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,635 DEBUG [ckan.logic] check access OK - ckanext_pages_update user=carlo
[Mon Jun 22 10:11:14.639263 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,639 DEBUG [ckan.lib.base] rendering /usr/lib/ckan/default/src/ckan/ckan/templates/snippets/home_breadcrumb_item.html [jinja2]
[Mon Jun 22 10:11:14.640425 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,640 DEBUG [ckan.lib.base] rendering /usr/lib/ckan/default/src/ckan/ckan/templates/package/snippets/resources.html [jinja2]
[Mon Jun 22 10:11:14.641786 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,641 DEBUG [ckan.lib.base] rendering /usr/lib/ckan/default/src/ckan/ckan/templates/package/snippets/resources.html [jinja2]
[Mon Jun 22 10:11:14.642352 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,642 DEBUG [ckan.lib.base] rendering /usr/lib/ckan/default/ckanext-datapackager/ckanext/datapackager/templates/datapackage/snippets/import_datapackage_form.html [jinja2]
[Mon Jun 22 10:11:14.643209 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,643 DEBUG [ckan.lib.base] rendering /usr/lib/ckan/default/ckanext-datapackager/ckanext/datapackager/templates/datapackage/snippets/import_datapackage_basic_fields.html [jinja2]
[Mon Jun 22 10:11:14.646658 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,646 DEBUG [ckan.logic] check access OK - organization_list_for_user user=carlo
[Mon Jun 22 10:11:14.652808 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,652 DEBUG [ckan.logic] check access OK - sysadmin user=carlo
[Mon Jun 22 10:11:14.653950 2020] [wsgi:error] [pid 9284:tid 139863850792704] 2020-06-22 10:11:14,653 INFO [ckan.lib.base] /import_datapackage render time 4.598 seconds
NOTE: I see same error reported here, not sure if it can help: https://github.com/geosolutions-it/ckanext-dcatapit/issues/171
Please preserve this line to notify @amercader (lead of this repository)
Hi, just installed also validator:
https://github.com/frictionlessdata/ckanext-validation
Still unable to import datapackages: https://datahub.io/core/covid-19/r/covid-19_zip.zip
Error:
AttributeError: 'Package' object has no attribute 'metadata'
Log
[Mon Jun 22 12:47:29.207463 2020] [wsgi:error] [pid 15090:tid 140576186115840] 2020-06-22 12:47:29,207 DEBUG [ckan.config.middleware.pylons_app] Pylons route match: {'action': u'import_datapackage', 'controller': u'ckanext.datapackager.controllers.datapackage:DataPackageController'} Origin: extension
[Mon Jun 22 12:47:29.207979 2020] [wsgi:error] [pid 15090:tid 140576186115840] 2020-06-22 12:47:29,207 DEBUG [ckan.config.middleware] Route support answers for POST /import_datapackage: [(False, 'flask_app'), (True, 'pylons_app', 'extension')]
[Mon Jun 22 12:47:29.208145 2020] [wsgi:error] [pid 15090:tid 140576186115840] 2020-06-22 12:47:29,208 DEBUG [ckan.config.middleware] Serving request via pylons_app app
[Mon Jun 22 12:47:29.222784 2020] [wsgi:error] [pid 15090:tid 140576186115840] 2020-06-22 12:47:29,222 DEBUG [ckan.logic] check access OK - package_create user=carlo
[Mon Jun 22 12:47:33.589111 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] Error - <type 'exceptions.AttributeError'>: 'Package' object has no attribute 'metadata'
[Mon Jun 22 12:47:33.591077 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] URL: http://data.review.fao.org/import_datapackage
[Mon Jun 22 12:47:33.591180 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/weberror/errormiddleware.py', line 171 in __call__
[Mon Jun 22 12:47:33.591232 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] app_iter = self.application(environ, sr_checker)
[Mon Jun 22 12:47:33.591276 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 147 in __call__
[Mon Jun 22 12:47:33.591323 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] resp = self.call_func(req, *args, **self.kwargs)
[Mon Jun 22 12:47:33.591364 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 208 in call_func
[Mon Jun 22 12:47:33.591419 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] return self.func(req, *args, **kwargs)
[Mon Jun 22 12:47:33.591465 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/fanstatic/publisher.py', line 234 in __call__
[Mon Jun 22 12:47:33.591507 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] return request.get_response(self.app)
[Mon Jun 22 12:47:33.591553 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1053 in get_response
[Mon Jun 22 12:47:33.591599 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] application, catch_exc_info=False)
[Mon Jun 22 12:47:33.591641 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1022 in call_application
[Mon Jun 22 12:47:33.591696 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] app_iter = application(self.environ, start_response)
[Mon Jun 22 12:47:33.591745 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 147 in __call__
[Mon Jun 22 12:47:33.591790 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] resp = self.call_func(req, *args, **self.kwargs)
[Mon Jun 22 12:47:33.591830 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/dec.py', line 208 in call_func
[Mon Jun 22 12:47:33.591870 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] return self.func(req, *args, **kwargs)
[Mon Jun 22 12:47:33.591909 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/fanstatic/injector.py', line 54 in __call__
[Mon Jun 22 12:47:33.591949 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] response = request.get_response(self.app)
[Mon Jun 22 12:47:33.591993 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1053 in get_response
[Mon Jun 22 12:47:33.592035 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] application, catch_exc_info=False)
[Mon Jun 22 12:47:33.592074 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/webob/request.py', line 1022 in call_application
[Mon Jun 22 12:47:33.592114 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] app_iter = application(self.environ, start_response)
[Mon Jun 22 12:47:33.592153 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/src/ckan/ckan/config/middleware/pylons_app.py', line 262 in inner
[Mon Jun 22 12:47:33.592197 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] result = application(environ, start_response)
[Mon Jun 22 12:47:33.592238 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/beaker/middleware.py', line 73 in __call__
[Mon Jun 22 12:47:33.592283 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] return self.app(environ, start_response)
[Mon Jun 22 12:47:33.592323 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/beaker/middleware.py', line 156 in __call__
[Mon Jun 22 12:47:33.592363 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] return self.wrap_app(environ, session_start_response)
[Mon Jun 22 12:47:33.592409 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/routes/middleware.py', line 131 in __call__
[Mon Jun 22 12:47:33.592455 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] response = self.app(environ, start_response)
[Mon Jun 22 12:47:33.592496 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/src/ckan/ckan/config/middleware/common_middleware.py', line 30 in __call__
[Mon Jun 22 12:47:33.592535 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] return self.app(environ, start_response)
[Mon Jun 22 12:47:33.592574 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/src/ckan/ckan/config/middleware/common_middleware.py', line 56 in __call__
[Mon Jun 22 12:47:33.592614 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] return self.app(environ, start_response)
[Mon Jun 22 12:47:33.592724 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/wsgiapp.py', line 125 in __call__
[Mon Jun 22 12:47:33.592766 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] response = self.dispatch(controller, environ, start_response)
[Mon Jun 22 12:47:33.592806 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/wsgiapp.py', line 324 in dispatch
[Mon Jun 22 12:47:33.592855 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] return controller(environ, start_response)
[Mon Jun 22 12:47:33.592897 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 240 in __call__
[Mon Jun 22 12:47:33.592937 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] res = WSGIController.__call__(self, environ, start_response)
[Mon Jun 22 12:47:33.592977 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 221 in __call__
[Mon Jun 22 12:47:33.593017 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] response = self._dispatch_call()
[Mon Jun 22 12:47:33.593057 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 172 in _dispatch_call
[Mon Jun 22 12:47:33.593096 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] response = self._inspect_call(func)
[Mon Jun 22 12:47:33.593136 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 107 in _inspect_call
[Mon Jun 22 12:47:33.593181 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] result = self._perform_call(func, args)
[Mon Jun 22 12:47:33.593224 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/controllers/core.py', line 60 in _perform_call
[Mon Jun 22 12:47:33.593273 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] return func(**args)
[Mon Jun 22 12:47:33.593316 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/ckanext/datapackager/controllers/datapackage.py', line 46 in import_datapackage
[Mon Jun 22 12:47:33.593362 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] params,
[Mon Jun 22 12:47:33.593411 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/src/ckan/ckan/logic/__init__.py', line 466 in wrapped
[Mon Jun 22 12:47:33.593455 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] result = _action(context, data_dict, **kw)
[Mon Jun 22 12:47:33.593499 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/ckanext/datapackager/logic/action/create.py', line 40 in package_create_from_datapackage
[Mon Jun 22 12:47:33.593543 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] dataset_dict = converter.datapackage_to_dataset(dp)
[Mon Jun 22 12:47:33.593586 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] File '/usr/lib/ckan/default/lib/python2.7/site-packages/ckanext/datapackager/lib/converter.py', line 91 in datapackage_to_dataset
[Mon Jun 22 12:47:33.593630 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] 'name': datapackage.metadata['name'].lower()
[Mon Jun 22 12:47:33.593673 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] AttributeError: 'Package' object has no attribute 'metadata'
[Mon Jun 22 12:47:33.593721 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991]
[Mon Jun 22 12:47:33.593764 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991]
[Mon Jun 22 12:47:33.593806 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] CGI Variables
[Mon Jun 22 12:47:33.593848 2020] [wsgi:error] [pid 15090:tid 140576186115840] [remote 127.0.0.1:47991] -------------