Add progress reporting where appropriate

[Arlodotexe]

trafficstars

Progress reporting is a good call out. Generally the library is built so that progress reporting can be implemented application-side during an operation (e.g. navigating directories, moving things around, stream transfers). However, there's a few long-running extension methods where we could consider an IProgress parameter, such as Copy and Move.

Here is the inbox implementation for copying a file into a folder. Presumably, dotnet should supply us with an overload that takes an IProgress in Stream.CopyToAsync, but that doesn't appear to be the case. We'd need to build this ourselves or use one of the many versions that others have built.

Originally posted by @Arlodotexe in https://github.com/Arlodotexe/OwlCore.Storage/issues/12#issuecomment-2160903901

[itsWindows11]

ChatGPT proposes this:

using System;
using System.IO;
using System.Threading;
using System.Threading.Tasks;

public async Task CopyStreamWithProgressAsync(
    Stream source, 
    Stream destination, 
    IProgress<double> progress = null, 
    CancellationToken cancellationToken = default)
{
    const int bufferSize = 81920; // 80 KB buffer
    byte[] buffer = new byte[bufferSize];
    long totalBytesRead = 0;
    long totalLength = source.Length;

    int bytesRead;
    while ((bytesRead = await source.ReadAsync(buffer, 0, buffer.Length, cancellationToken)) > 0)
    {
        await destination.WriteAsync(buffer, 0, bytesRead, cancellationToken);
        totalBytesRead += bytesRead;
        
        if (progress != null && totalLength > 0)
        {
            double percentage = (double)totalBytesRead / totalLength * 100;
            progress.Report(percentage);
        }
    }
}

Though we need to report the total bytes read along with the length instead of the percentage, and leave the rest for the implementor to decide what they want to do with the progress, and use ArrayPool. No idea if some underlying stream implementations will require special handling here.

[Arlodotexe]

Yeah, we should look at what others have already brainstormed online on sites like StackOverflow. Figuring out exactly what <T> to use for IProgress<T> isn't straightforward.

[Arlodotexe]

We also need to expose the bufferSize parameter in CreateCopyOfAsync (interface, and extension and fallback delegate). Since they're similar breaking changes, they should ship together.

Need to prototype a few attempts at progress reporting before we can land on what to use for the T in IProgress<T> here.

[yoshiask]

In Fluent Store, I have an IProgress<double?> implementation named DataTransferProgress:

• The basic progress is double, between 0.0 and 1.0. Makes it easy to use in UI like progress bars
• This double is nullable to handle cases where percentage progress is unavailable, for example when the length of the source stream is unknown. Allows for UI to show an indeterminant progress bar rather than jumping from 0 to 100%
• Additional 'raw' information, the number of bytes downloaded and the total number of bytes. Gives consumers the opportunity to report progress more granularly or guess download speeds.

My implementation works but is a bit odd because it implements IProgress<T> and puts state variable in the progress class. I believe we should put the additional information in a record and use the whole thing as the progress type, something like this:

public record DataTransferProgressState(ulong BytesDownloaded, ulong? TotalBytes = null)
{
    public double? PercentageComplete => TotalBytes is not null
        ? BytesDownloaded / TotalBytes
        : null;
}

public class SampleProgress : IProgress<DataTransferProgressState>
{
    // ....
}

long and ulong are certainly large enough for reporting file lengths in bytes. The max value for a signed 64-bit integer is 9,223 petabytes, and unsigned 64-bit is double that. uint is too small however, at only around 4 gigabytes.