django-storages icon indicating copy to clipboard operation
django-storages copied to clipboard

Unable to upload files/images to cloud storage (GCP) with Django

Open jeetpatel9 opened this issue 4 years ago • 3 comments

I am trying to upload files & images to the GCP bucket which I created a while ago. I am using Django rest-framework as the backend. I came across this library called [django storages][1]. I did everything stated in the library documentation.

  1. Create a service account, provided permission as storage admin and generated the json key.

  2. In my projects settings.py file I have declared these attributes -

DEFAULT_FILE_STORAGE = "storages.backends.gcloud.GoogleCloudStorage"

GS_BUCKET_NAME = os.environ.get("GS_BUCKET_NAME")

GS_CREDENTIALS = service_account.Credentials.from_service_account_file("ibdax.json")

GS_PROJECT_ID="project-id"

Additionally, my models look exactly the same as said in the documentation.

But I still get this error -

{
    "error": "HTTPSConnectionPool(host='oauth2.googleapis.com', port=443): Max retries exceeded with url: /token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0xffffa84881d0>: Failed to establish a new connection: [Errno 111] Connection refused'))"
}

These are the logs -

Internal Server Error: /api/properties/create-property/
ibdax               | Traceback (most recent call last):
ibdax               |   File "/usr/local/lib/python3.7/site-packages/asgiref/sync.py", line 482, in thread_handler
ibdax               |     raise exc_info[1]
ibdax               |   File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 38, in inner
ibdax               |     response = await get_response(request)
ibdax               |   File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 138, in __acall__
ibdax               |     )(request, response)
ibdax               |   File "/usr/local/lib/python3.7/site-packages/asgiref/sync.py", line 444, in __call__
ibdax               |     ret = await asyncio.wait_for(future, timeout=None)
ibdax               |   File "/usr/local/lib/python3.7/asyncio/tasks.py", line 414, in wait_for
ibdax               |     return await fut
ibdax               | concurrent.futures._base.CancelledError
ibdax               | Application instance <Task pending coro=<StaticFilesWrapper.__call__() running at /usr/local/lib/python3.7/site-packages/channels/staticfiles.py:44> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/local/lib/python3.7/asyncio/futures.py:351, <TaskWakeupMethWrapper object at 0xffff8afea050>()]>> for connection <WebRequest at 0xffff8b38d810 method=POST uri=/api/properties/create-property/?property_files[O]&property_images[O]image: clientproto=HTTP/1.1> took too long to shut down and was killed.

After a long time API return this error response. Where am I going wrong.

jeetpatel9 avatar Feb 12 '22 12:02 jeetpatel9

@jeetpatel9 where are you running this code? Locally or inside Google Cloud.

What code is running? What is in api/properties/create-property

The first error is your code unable to hit a Google server. It may be a temporary problem.

I use the library with django-storages and Django Rest Framework successfully.

sww314 avatar Feb 13 '22 16:02 sww314

@sww314 I have been running this inside a docker container on mac with an m1 chip and I also tried it running locally outside of the docker container.

api/properties/create-property This is a Django API endpoint that accepts multipart form data. This API is basically responsible for uploading the images/files to the google cloud bucket.

Also, I have been stuck on this for a long now.

jeetpatel9 avatar Feb 14 '22 08:02 jeetpatel9

@jeetpatel9 sorry I just saw this.

I would test out the connection outside of the request. Something is keeping you from connecting to google. The timeout maybe a good indicator that all the network connections are timing out.

I run my config in a very similar manner. Our dev teams, use a docker container on (Intel based) Mac that has no problem connecting to Google.

sww314 avatar May 02 '22 23:05 sww314