High nginx cpu usage on upload

[jbd]

Hi,

I'm using nginx 1.8.0 with the 2.2 branch of nginx-upload-module. I'm using the provided nginx configuration and upload static page. I've just altered the client_max_body_size directive to allow big upload.

I have created a 10G file using dd, and when I'm uploading it using curl, nginx takes 100% of cpu :

curl -F file=@/tmp/10G http://localhost:8000/upload

The upload took 2 minutes, that is 85 MBytes/s, which is only 2 times slower than a simple cp (cold cache) but the cp use only 2% of the cpu.

Is this a normal behaviour ?

[kminooie]

Hi, I was just wondering, do use any of the checksums? or turn on the resumable upload feature?

[jbd]

I've ran the test again without using checksums in the aggregate form fields, and I've got the same behaviour :

upload_set_form_field $upload_field_name '{"filename":"$upload_file_name", "content_type": "$upload_content_type", "path": "$upload_tmp_path"}';

Do you observe the same thing on your side ?

[vkholodkov]

Consider extending upload buffer size:

upload_buffer_size 4M;

I this doesn't help, we need to dig further.

[kminooie]

I haven't been using it with such large files, but I am planing to do so for files of up to a few hundreds MB and your issue got my attention. I will start testing it tomorrow, and I will get back to you. meanwhile, as @vkholodkov just pointed out, upload_buffer_size could be the reason. nginx buffer size by default is relatively small, and, I am guessing, a small buffer size could cause a lot of context switch and that can explain the high CPU load.