[Improvement] Use different cURL option to prevent OOM error in collector

[ejweber]

A Longhorn user in https://github.com/longhorn/longhorn/issues/5985 noticed that the support bundle collection process could not complete on resource constrained hardware (Raspberry Pi 4s). They found the following in the support bundle agent logs:

+ curl -v -i -H 'Content-Type: application/zip' --data-binary @node_bundle.zip http://10.244.4.60:8080/nodes/w1
curl: option --data-binary: out of memory
curl: try 'curl --help' or 'curl --manual' for more information

This seems to be a known issue with the cURL --data-binary flag. The entire contents of a file are loaded into RAM before the file is sent. A number of (admittedly old) GitHub and StackOverflow issues mention using the -T(--upload-file) option as an alternative.

If it is possible, we should consider using the -T(--upload-file) option to ensure bundles upload successfully even when memory is limited.

https://github.com/curl/curl/issues/1385 https://github.com/curl/curl/issues/290 https://stackoverflow.com/a/69788901

[ejweber]

The user posted an update showing that approximately 7 GB of syslog files were collected to be sent during the failure. Maybe instead (or in addition) we should limit the size of the files we are sending?

https://github.com/longhorn/longhorn/issues/5985#issuecomment-1562249250

[samip5]

I would also like to point out that sometimes those log files can be 10GB+, in my new case, 20GB for syslog and messages logs...