Intermittent 504 errors when uploading image

[mstockd]

Lately we are seeing intermittent 504 status errors when using this resource in a put step, trying to push an image built with the concourse/builder-task resource. This happens in ~ one out of 30 builds. Currently we resort to re-triggering the build and will probably use the attempt step modifier to get around this, but it would be great if the retry logic could be built-in / we could identify the cause and fix it for good.

Specifically we are using the forked resource with ECR support linked in here: https://github.com/concourse/registry-image-resource/pull/27

The concourse workers are running inside EC2 as well, so I assume networking not to be an issue.

This is what we see in the logs (debug is not yet enabled):

INFO[0009] pushing sha256:34a09a2c3c928632ba4744838ae799de33f85a70887eef8afd8080af6fe0a993 to <ECR-registry>/<image>:<tag>
2019/08/23 00:22:46 existing blob: sha256:37a3ead0766dcfba55ff2ccfa9860d25ee131493acebd7edc9eaae745c32411b
2019/08/23 00:22:46 existing blob: sha256:e53c3bcf61e07a59d01e2165e114c04160f066f2ed072da4c8cf91704e819179
2019/08/23 00:22:46 existing blob: sha256:dad366dafa97989db13e3371c5ade90997dc4a32b23e04617de5bbfeb6208b7c
2019/08/23 00:22:46 existing blob: sha256:0a4690c5d889e116874bf45dc757b515565a3bd9b0f6c04054d62280bb4f4ecf
2019/08/23 00:22:46 existing blob: sha256:687121a174978dda4981b1d1d0c047148b64cf6962df0edf440926cdc68074f0
2019/08/23 00:22:46 existing blob: sha256:d4542771caf6b5e86dce5c2a145a07f71af9f10e662f96c18c1a2547cae11651
2019/08/23 00:22:46 existing blob: sha256:c9cb2749d092c7fea537b446059e678c9ae92bfdc8beff186ee0cf84125a2027
2019/08/23 00:22:46 existing blob: sha256:e011ac85ea7cee3f26fbb106851eabcab8c9cd851badaf2b74797b1d2ed46b06
2019/08/23 00:22:46 existing blob: sha256:f0234a3d453d84d2fdb1cee057eed3408d1302a3cc6682afa9aca065277a03be
2019/08/23 00:22:46 existing blob: sha256:83ccbf5da27b43ce99e8cab70b6f18701bacd847b4047793ea939304faff3ff9
2019/08/23 00:22:46 existing blob: sha256:068294ceaa2e10baa1ae378b2ed47fe5ef9eef9e26cee7e57d319857d61bc4cd
2019/08/23 00:22:46 existing blob: sha256:2a31688e46c5c209e020965440e6d7ab15e1f6c1e71b8f1c21b788ed0454fffd
2019/08/23 00:24:12 pushed blob: sha256:06ff1ba3ace5a790b243100c5d1714801250c40df9b8854f7e6ab1211230b803
ERRO[0147] failed to upload image: unsupported status code 504; body: 

[xtremerui]

Is this still an issue or could I close it?