For no obvious reasons, my request hangs when I try to upload hundreds of images/files to aws

[tanweerdev]

Environment

• Elixir & Erlang versions (elixir --version): Erlang/OTP 22 [erts-10.7.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] Elixir 1.10.2 (compiled with Erlang/OTP 21)

• ExAws version mix deps |grep ex_aws: ex_aws 2.1.3 (Hex package) (mix) locked at 2.1.3 (ex_aws) 0bdbe2ae

• HTTP client version. IE for hackney do mix deps | grep hackney: hackney 1.16.0 (Hex package) (rebar3) locked at 1.16.0 (hackney) 3bf0bebb

Current behavior

Request hangs control gained back is not printed(when I try to upload multiple files total size bigger than 20 mb) But I see below error in my console. it means something went wrong during uploading files but nothing is there on the console

[warn] ExAws: HTTP ERROR: :closed for URL: "https://logs.us-east-2.amazonaws.com/" ATTEMPT: 1

I use below code to upload files

res =  files_list
    |> Task.async_stream(&upload_file(&1, images_upload), max_concurrency: 4, timeout: 1_500_000)
    |> Stream.run()
IO.inspect "control gained back"
res

and

def upload_file({file_type, local_file_path, file_name}, %ImagesUpload{} = images_upload) do
    file = File.read!(local_file_path)
    destination_path = Path.join([images_upload.upload_id, @file_types[file_type], file_name])

    S3.put_object(component_bucket(), destination_path, file)
    |> ExAws.request!()

    Logger.info("uploaded image #{destination_path}")
    {:ok, destination_path}
end

and in postman I get msg

Could not get response
Error: socket hang up

Expected behavior

It should give back control after success/failure response. Further details can be found here: https://elixirforum.com/t/for-no-obvious-reasons-my-request-hangs-when-i-try-to-upload-hundreds-of-images-files-to-aws/33405

[tanweerdev]

Note: it works for smaller files (total fille size) of upto 10-15 mb

[benwilson512]

https://github.com/ex-aws/ex_aws#deprecation-notice