FluentStorage icon indicating copy to clipboard operation
FluentStorage copied to clipboard

FluentStorage.Azure.Blobs.WriteAsync does not support "append=true"

Open gentledepp opened this issue 1 year ago • 2 comments

Even though the method signature suggests it, AzureBlobStorages

public async Task WriteAsync(
      string fullPath,
      Stream dataStream,
      bool append = false,
      CancellationToken cancellationToken = default (CancellationToken))

setting append to true still just overwrites any preexisting blob.

This is because it uses BlockBlobClient.UploadAsync internally, where the documentation clearly states:

The UploadAsync(Stream, BlobUploadOptions, CancellationToken) operation overwrites the contents of the blob, creating a new block blob if none exists. Overwriting an existing block blob replaces any existing metadata on the blob.

gentledepp avatar Sep 18 '24 11:09 gentledepp

Note. Naively using the OpenWriteAsync API that - by its signature - supports appending to an existing blockblob like so:

	public async Task WriteAsync(string fullPath, Stream dataStream,
		   bool append = false, CancellationToken cancellationToken = default) {
			GenericValidation.CheckBlobFullPath(fullPath);

			if (dataStream == null)
				throw new ArgumentNullException(nameof(dataStream));

			(BlobContainerClient container, string path) = await GetPartsAsync(fullPath, true).ConfigureAwait(false);

			BlockBlobClient client = container.GetBlockBlobClient(path);

			try {
				var targetStream = await client
					.OpenWriteAsync(!append, new BlockBlobOpenWriteOptions(), cancellationToken: cancellationToken)
					.ConfigureAwait(false);
				await dataStream.CopyToAsync(targetStream, 4096, cancellationToken).ConfigureAwait(false);
				// await client.UploadAsync(
				//    new StorageSourceStream(dataStream),
				//    cancellationToken: cancellationToken).ConfigureAwait(false);
			}
			catch (RequestFailedException ex) when (ex.ErrorCode == "OperationNotAllowedInCurrentState") {
				//happens when trying to write to a non-file object i.e. folder
			}
		}


throws an Exception

System.ArgumentException : BlockBlobClient.OpenWrite only supports overwriting bei Azure.Storage.Blobs.Specialized.BlockBlobClient.<OpenWriteInternal>d__58.MoveNext()

So this is a dead end too...

gentledepp avatar Sep 18 '24 12:09 gentledepp

Since we have potentially huge files we need to stream to Azure Blobstorage, the only viable alternative I found was to extend the IAzureBlobStorage interface with a new method Task<Stream> OpenWriteAsync(string fullPath, CancellationToken cancellationToken = default);

gentledepp avatar Sep 18 '24 12:09 gentledepp

Closing as PR is accepted.

robinrodricks avatar Oct 07 '24 12:10 robinrodricks