ckanext-dcat icon indicating copy to clipboard operation
ckanext-dcat copied to clipboard

Harvester crashes with missing title

Open jze opened this issue 1 year ago • 1 comments

I had a harvester run abort today with the following error message:

[ckanext.harvest.model] Error when processsing dataset: KeyError('title') / Traceback (most recent call last):
  File "/usr/lib/ckan/default/src/ckanext-dcat/ckanext/dcat/harvesters/rdf.py", line 218, in gather_stage
    dataset['name'] = self._gen_new_name(dataset['title'])
KeyError: 'title'

I have not yet figured out which record contains the error and what it looks like. However, the harvest run should not crash completely due to a single incorrect dataset.

jze avatar Jul 31 '23 05:07 jze

Thanks @jze . Yes, of course. I agree. Having a try/except around each dataset or checking for an existent title and creating a gather_error would be the better way in this case. Would you mind submitting a PR with the changes needed?

seitenbau-govdata avatar Aug 01 '23 13:08 seitenbau-govdata