datapusher icon indicating copy to clipboard operation
datapusher copied to clipboard

Datapusher settings

Open carlosips opened this issue 6 years ago • 7 comments

Hi,

I'm trying to change MAX_CONTENT_LENGTH parameter to 15728640, to editing file in /etc/ckan/datapusher_settings.py, append MAX_CONTENT_LENGTH=15728640 and restarting apache, but when I upload a file to datastore, ckan shows me next message:

Error: Resource too large to process: 10487680 > max (10485760).

Anybody knows what I'm doing wrong?

P.D: I'm using Ckan version 2.7

carlosips avatar Nov 27 '18 15:11 carlosips

@carlosips You may want to take a look at the 2.7 docs for config options.

You have to adjust the max file size in your production.ini (or development.ini) file. The default is 1048576 1(MB) there.

boykoc avatar Nov 27 '18 21:11 boykoc

@boykoc ckan.resource_proxy.max_file_size parameter is used to perform in-line preview of file. My question is about an error uploading a file to datastore. You can see error message generation here:

https://github.com/ckan/datapusher/blob/master/datapusher/jobs.py#L378

I have modified jobs.py hardcoding MAX_CONTENT_LENGTH value (for testing purpose) and it works fine.

carlosips avatar Nov 28 '18 15:11 carlosips

@carlosips my bad, I was thinking the ckan.max_resource_size but you're right, that raises a different error.

I've tested this on my setup (apache, nginx, ckan 2.8.1) and have it working with a 26MB (26000000 bytes) limit. I followed the default deploy instructions.

I then updated the /etc/ckan/datapusher_settings.py by adding MAX_CONTENT_LENGTH=26000000 as a new line. I also updated the ckan.max_resource_size=26 in my production.ini to allow a larger resource. After restarting apache it seems to be working fine.

Sorry not more help.

boykoc avatar Nov 28 '18 17:11 boykoc

@boykoc thanks. I keep investigating

carlosips avatar Nov 29 '18 16:11 carlosips

facing the same here... found something @carlosips ?

blavoie avatar Mar 20 '19 16:03 blavoie

Nothing useful. Im sorry

carlosips avatar Mar 21 '19 11:03 carlosips

is this the ckan or datapusher problem? I'm having the same.

namgk avatar Apr 25 '19 13:04 namgk