Problem uploading empty files with upload_object

[dvdrbn-m9]

Summary

Using upload_object to upload an empty file returns response: 400 - Bad Request: too many Host headers

Detailed Information

Python 3.6.8 Libcloud: 3.3.1 OS: rhel 7.9

I use libcloud to upload objects to 3 providers - aws s3, minio & netapp storageGrid, I use the s3 driver for all of them. upload_object function works fine except when I try to upload an empty file (I sometimes need to store just meta-data with a specific object name for later use). When trying to upload an empty file I get the response:

status: 400
reason: 'Bad Request: too many Host headers'

If I add anything to the file's content it uploads successfully with the meta-data, so its not a problem with permissions/path in s3 or something like that. I also tried without the meta-data, but that didn't work either. I tried using upload_object_via_stream and found the following code works:

with io.BytesIO() as iterator:
    driver.upload_object_via_stream(iterator=iterator,
                                    container=container,
                                    object_name=objName,
                                    extra=extra)

This code uploads a 0 bytes file successfully to aws s3 & minio, but not to netapp's storageGrid (so unfortunately I can't use it). I think it's a limitation of storageGrid's multi-part upload and not some other storage restriction (we have existing empty objects on storageGrid, but those were uploaded using jclouds).

I think using upload_object will be the best solution for all 3 providers, but I don't understand why it's not uploading empty files and would appreciate help with this. Thanks!

Reproduce

1. touch ~/test-empty.txt
2. Use upload_object to upload test-empty.txt

[stale[bot]]

Thanks for contributing to this issue. As it has been 90 days since the last activity, we are automatically marking is as stale. If this issue is not relevant or applicable anymore (problem has been fixed in a new version or similar), please close the issue or let us know so we can close it. On the contrary, if the issue is still relevant, there is nothing you need to do, but if you have any additional details or context which would help us when working on this issue, please include it as a comment to this issue.

[thomas-riccardi]

We encountered this issue too (incoming GCP change: they'll stop supporting multiple Host headers requests, and return 400).

We tracked down the issue to requests: https://github.com/psf/requests/pull/5391 : it was fixed in requests 2.27.0.