Add more verbose output mode (progress, upload speed, etc.)

[gdubicki]

TL;DR

I would like to be able to see the progress of the upload during it and its effective speed in MB/s at the end.

Detailed design

The actions/upload-artifacts@v4 shows this kind of output:

Wed, 22 May 2024 10:57:16 GMT With the provided path, there will be 1 file uploaded
Wed, 22 May 2024 10:57:16 GMT Artifact name is valid!
Wed, 22 May 2024 10:57:16 GMT Root directory input is valid!
Wed, 22 May 2024 10:57:16 GMT Beginning upload of artifact content to blob storage
Wed, 22 May 2024 10:57:24 GMT Uploaded bytes 8388608
Wed, 22 May 2024 10:57:30 GMT Uploaded bytes 16777216
Wed, 22 May 2024 10:57:36 GMT Uploaded bytes 25165824
(...)
Wed, 22 May 2024 11:36:57 GMT Uploaded bytes 3003121664
Wed, 22 May 2024 11:37:03 GMT Uploaded bytes 3011510272
Wed, 22 May 2024 11:37:06 GMT Uploaded bytes 3015977440
Wed, 22 May 2024 11:37:06 GMT Finished uploading artifact content to blob storage!
Wed, 22 May 2024 11:37:06 GMT SHA256 hash of uploaded artifact zip is <redacted>
Wed, 22 May 2024 11:37:06 GMT Finalizing artifact upload
Wed, 22 May 2024 11:37:06 GMT Artifact XXX successfully finalized. Artifact ID <redacted>
Wed, 22 May 2024 11:37:06 GMT Artifact YYY has been successfully uploaded! Final size is 3015977440 bytes. Artifact ID is <redacted>
Wed, 22 May 2024 11:37:06 GMT Artifact download URL: https://github.com/<redacted>

I personally feel this is oververbose and would prefer something like this:

Wed, 22 May 2024 10:57:16 GMT Total size to upload is: 3.016 GiB
Wed, 22 May 2024 10:57:16 GMT Beginning upload to GCS
Wed, 22 May 2024 10:57:24 GMT Uploaded 30.16 MiB (1%)
Wed, 22 May 2024 10:57:30 GMT Uploaded 60.32 MiB (2%)
(...)
Wed, 22 May 2024 11:36:57 GMT Uploaded 2.96 GiB (98%)
Wed, 22 May 2024 11:37:03 GMT Uploaded 2.99 GiB (99%)
Wed, 22 May 2024 11:37:06 GMT Finished uploading to GCS!
Wed, 22 May 2024 11:37:06 GMT Upload speed: 125 MiB/s

In terms of the units I think that because this output is for human consumption only, we should show the data sizes in either only percentages or in the most human-readable units and percentages.

I think that we should use data units based on the power of 2 (MiB, GiB, etc.) as they are used by gcloud cp when downloading files:

Copying gs://<redacted> to file://<redacted>
................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Average throughput: 599.5MiB/s

Additional information

This feature would allow troubleshooting too slow uploads, for example with self-hosted runners running in GCP, the same region as the GCS bucket.

[sethvargo]

Hi @gdubicki - what problem are you trying to solve here? If you enable GitHub Actions debug logging, you can see each file get uploaded along with timestamps, but that level of verbosity seems unnecessary for most use.

[gdubicki]

Hi @gdubicki - what problem are you trying to solve here?

Hi @sethvargo, I somehow missed your reply here, sorry for that.

I ended up with using a script with gcloud cp because I think I had a problem with what I mentioned in the description:

too slow uploads, for example with self-hosted runners running in GCP, the same region as the GCS bucket.

I didn’t try:

If you enable GitHub Actions debug logging, you can see each file get uploaded along with timestamps, but that level of verbosity seems unnecessary for most use.

…but I assume it would mean I would have to count the upload speed myself then, which I think is something the action should do.