datapusher icon indicating copy to clipboard operation
datapusher copied to clipboard

error while adding csv-files with duplicate column names

Open solvistas-hartmi opened this issue 11 years ago • 3 comments

Currently, it is not possible to add csv-files with duplicate column names, since these are also used as table-columns in postgres. In web the datapusher returns an "internal server error 500" at /api/3/action/datastore_create.

solvistas-hartmi avatar Dec 15 '14 17:12 solvistas-hartmi

We discussed this at the meeting, and the general consensus was that although it might be possible to append a number to duplicate columns, it would introduce problems later in the process.

The best solution currently is to make sure there is a usable error message so that the user can correct the mistake and re-upload the file. This isn't to say it'll never handle it, but in the foreseeable future this is the safest approach (unless a PR appears from somewhere).

rossjones avatar Dec 16 '14 12:12 rossjones

same issue with CKAN 2.4.1 . It would be great to get at least an error message pointing the real cause. now I get only:

[Mon Nov 09 11:21:44 2015] [error] CKAN DataStore bad response. Status code: 500 Internal Server Error. At: http://localhost/api/3/action/datastore_create.
[Mon Nov 09 11:21:44 2015] [error] Job "push_to_datastore (trigger: RunTriggerNow, run = True, next run at: None)" raised an exception
[Mon Nov 09 11:21:44 2015] [error] Traceback (most recent call last):
[Mon Nov 09 11:21:44 2015] [error]   File "/usr/lib/ckan/datapusher/lib/python2.7/site-packages/apscheduler/scheduler.py", line 512, in _run_job
[Mon Nov 09 11:21:44 2015] [error]     retval = job.func(*job.args, **job.kwargs)
[Mon Nov 09 11:21:44 2015] [error]   File "/usr/lib/ckan/datapusher/src/datapusher/datapusher/jobs.py", line 389, in push_to_datastore
[Mon Nov 09 11:21:44 2015] [error]     records, api_key, ckan_url)
[Mon Nov 09 11:21:44 2015] [error]   File "/usr/lib/ckan/datapusher/src/datapusher/datapusher/jobs.py", line 205, in send_resource_to_datastore
[Mon Nov 09 11:21:44 2015] [error]     check_response(r, url, 'CKAN DataStore')
[Mon Nov 09 11:21:44 2015] [error]   File "/usr/lib/ckan/datapusher/src/datapusher/datapusher/jobs.py", line 146, in check_response
[Mon Nov 09 11:21:44 2015] [error]     response=response.text)
[Mon Nov 09 11:21:44 2015] [error] HTTPError

Thanks

arieljlira avatar Nov 09 '15 14:11 arieljlira

Current Ckan (2.8) throws the right Error: Response: {"help": "http://h2893531.stratoserver.net/api/3/action/help_show?name=datastore_create", "success": false, "error": {"field": ["Duplicate column names are not supported"], "__type": "Validation Error...

Just encountered it. This may be safe to close - consensus on keeping it and correct error message are sufficient requirements in general to close issues - just my 2ct.

erdnuesse avatar Jul 19 '20 11:07 erdnuesse