upload-cloud-storage icon indicating copy to clipboard operation
upload-cloud-storage copied to clipboard

Add more verbose output mode (progress, upload speed, etc.)

Open gdubicki opened this issue 9 months ago • 1 comments

TL;DR

I would like to be able to see the progress of the upload during it and its effective speed in MB/s at the end.

Detailed design

The actions/upload-artifacts@v4 shows this kind of output:

Wed, 22 May 2024 10:57:16 GMT With the provided path, there will be 1 file uploaded
Wed, 22 May 2024 10:57:16 GMT Artifact name is valid!
Wed, 22 May 2024 10:57:16 GMT Root directory input is valid!
Wed, 22 May 2024 10:57:16 GMT Beginning upload of artifact content to blob storage
Wed, 22 May 2024 10:57:24 GMT Uploaded bytes 8388608
Wed, 22 May 2024 10:57:30 GMT Uploaded bytes 16777216
Wed, 22 May 2024 10:57:36 GMT Uploaded bytes 25165824
(...)
Wed, 22 May 2024 11:36:57 GMT Uploaded bytes 3003121664
Wed, 22 May 2024 11:37:03 GMT Uploaded bytes 3011510272
Wed, 22 May 2024 11:37:06 GMT Uploaded bytes 3015977440
Wed, 22 May 2024 11:37:06 GMT Finished uploading artifact content to blob storage!
Wed, 22 May 2024 11:37:06 GMT SHA256 hash of uploaded artifact zip is <redacted>
Wed, 22 May 2024 11:37:06 GMT Finalizing artifact upload
Wed, 22 May 2024 11:37:06 GMT Artifact XXX successfully finalized. Artifact ID <redacted>
Wed, 22 May 2024 11:37:06 GMT Artifact YYY has been successfully uploaded! Final size is 3015977440 bytes. Artifact ID is <redacted>
Wed, 22 May 2024 11:37:06 GMT Artifact download URL: https://github.com/<redacted>

I personally feel this is oververbose and would prefer something like this:

Wed, 22 May 2024 10:57:16 GMT Total size to upload is: 3.016 GiB
Wed, 22 May 2024 10:57:16 GMT Beginning upload to GCS
Wed, 22 May 2024 10:57:24 GMT Uploaded 30.16 MiB (1%)
Wed, 22 May 2024 10:57:30 GMT Uploaded 60.32 MiB (2%)
(...)
Wed, 22 May 2024 11:36:57 GMT Uploaded 2.96 GiB (98%)
Wed, 22 May 2024 11:37:03 GMT Uploaded 2.99 GiB (99%)
Wed, 22 May 2024 11:37:06 GMT Finished uploading to GCS!
Wed, 22 May 2024 11:37:06 GMT Upload speed: 125 MiB/s

In terms of the units I think that because this output is for human consumption only, we should show the data sizes in either only percentages or in the most human-readable units and percentages.

I think that we should use data units based on the power of 2 (MiB, GiB, etc.) as they are used by gcloud cp when downloading files:

Copying gs://<redacted> to file://<redacted>
................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Average throughput: 599.5MiB/s

Additional information

This feature would allow troubleshooting too slow uploads, for example with self-hosted runners running in GCP, the same region as the GCS bucket.

gdubicki avatar May 23 '24 07:05 gdubicki