Add support for uploading objects to File Shares

[bbi-willemtoorenburgh]

Hello!

• Vote on this issue by adding a 👍 reaction
• If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

Hey there! This is my first foray into C#, .Net, and Pulumi, so please bear with me if I am missing some obvious things.

I'm trying to instrument some containers in Azure Container Apps, one of which is Vector. In order to configure Vector, I need to mount an Azure File Share to the container, which contains its config file. I saw that Pulumi has providers for both File Share and individual Blobs, so I figured that would be the best way to send my Vector config to the newly-created File Share, like so:

using Pulumi.AzureNative.Storage;

var fileShare = new FileShare("file-share-vector", new FileShareArgs
{
    ResourceGroupName = resourceGroup.Name,
    AccountName = storageAccount.Name,
    EnabledProtocols = EnabledProtocols.SMB,
    // Unit size is gigabytes
    ShareQuota = 1
});

var blobVectorConfig = new Blob("vector-config", new BlobArgs
{
    ResourceGroupName = resourceGroup.Name,
    AccountName = storageAccount.Name,
    ContainerName = fileShareVector.Name,
    BlobName = "vector.toml",
    Source = new FileAsset("./vector.toml")
});

Unfortunately, I get an error when I try to do so:

Diagnostics:
  azure-native:storage:Blob (vector-config):
    error: blobs.Client#PutBlockBlob: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ContainerNotFound" Message="The specified container does not exist.\nRequestId:<id>\nTime:2022-04-20T19:23:43.2117256Z"

After searching around the Storage section of the Azure Native provider docs, I couldn't find anything that indicates support for uploading objects to File Shares, only to Blob Containers. I must assume that there is some way to do this, as both the Azure Powershell module and the Azure CLI support uploading directly to fileshares.

My suggested course of action would be either:

1. Update the Blob resource's BlobArgs to support pointing ContainerName to a FileShare resource (should be non-breaking), or
2. Add a new input to BlobArgs to specify what kind of storage is being targeted (like ContainerType = ContainerTypes.BlobContainer | ContainerTypes.FileShare) (should also be non-breaking, if ContainerType has a default value of ContainerTypes.BlobContainer).

Option 2 is likely to be more future-friendly, in the event that Azure gets new kinds of storage services.

If there's already a way to upload something to a File Share, I'll gladly use that instead. Cheers!

Affected area/feature

Presumably this lives solely inside Pulumi.AzureNative.Storage.

[bbi-willemtoorenburgh]

For anyone in a similar situation, a colleague and I put together our own file upload implementation leveraging the Azure dotNet SDK. I don't know how to make Pulumi aware of it as a resource (and custom resources/providers are not supported for dotNet as of writing), and I'm sure there's a more elegant way to write this, but the file does get uploaded properly.

using Pulumi;
using Azure.Storage;
using Azure.Storage.Files.Shares;

class MyStack : Stack
{

    public MyStack()
    {
        var vectorConfigUploadTask = Output.Tuple(storageAccount.PrimaryEndpoints, storageAccount.Name, PrimaryStorageKey, fileShareVector.Name).Apply(async (outputs) =>
        {
            await UploadFileToFileShareAsync(
                outputs.Item1.File,
                outputs.Item2,
                outputs.Item3,
                outputs.Item4,
                "./vector.toml"
            );
        });
    }

    private static async Task UploadFileToFileShareAsync(string storageAccountUri, string storageAccountName, string storageAccountPrimaryKey, string shareName, string localFilePath, string destinationPath = "/")
    {
        var connectionUri = new ShareUriBuilder(new System.Uri(storageAccountUri)).ToUri();

        var shareServiceClient = new ShareServiceClient(
            connectionUri,
            new StorageSharedKeyCredential(storageAccountName, storageAccountPrimaryKey),
            default
        );

        var shareClient = shareServiceClient.GetShareClient(shareName);
        var directoryClient = destinationPath switch
        {
            "/" => shareClient.GetRootDirectoryClient(),
            _ => shareClient.GetDirectoryClient(destinationPath)
        };
        var fileName = System.IO.Path.GetFileName(localFilePath);
        var shareFileClient = directoryClient.GetFileClient(fileName);

        await using (var stream = System.IO.File.OpenRead(System.IO.Path.GetFullPath(localFilePath)))
        {
            await shareFileClient.CreateAsync(stream.Length);
            await shareFileClient.UploadRangeAsync(
                new Azure.HttpRange(0, stream.Length),
                stream
            );
        }
    }
}

[baartch]

Thanks for sharing your solution. This helped me creating the same for TypeScript.

The NPM module to install: https://www.npmjs.com/package/@azure/storage-file-share

The upload function

export async function uploadFileToAzureFileShare(
  account: string,
  accountKey: string,
  share: string,
  directory: string,
  filename: string,
  localFilePath: string
) {
  // Read the file content from the local file
  let content: string;
  try {
    content = fs.readFileSync(localFilePath, 'utf8');
  } catch (err) {
    console.error(err);
    throw err;
  }

  const credential = new StorageSharedKeyCredential(account, accountKey);
  const serviceClient = new ShareServiceClient(`https://${account}.file.core.windows.net`, credential);

  const shareClient = serviceClient.getShareClient(share);

  const directoryClient =
    directory === '/' ? shareClient.rootDirectoryClient : shareClient.getDirectoryClient(directory);
  const fileClient = directoryClient.getFileClient(filename);

  await fileClient.create(content.length);

  // Upload file range
  await fileClient.uploadRange(content, 0, content.length);
  pulumi.log.info(`File upload ${filename} successful`);
}

And here is an example on calling the function, after the file share is created:

    // Create a file share on the storage account for the Nginx configuration
    this.fsNginxConf = new storage.FileShare(
      getName('fs', 1),
      {
        resourceGroupName,
        accountName: this.storageAccount.name,
      },
      { parent: this }
    );

    // Upload the Nginx configuration file to the file share
    if (!pulumi.runtime.isDryRun()) {
      pulumi
        .all([this.storageAccount.name, this.getPrimaryStorageKey(), this.fsNginxConf.name])
        .apply(([accountName, accountKey, fsNginx]) => {
          uploadFileToAzureFileShare(
            accountName,
            accountKey,
            fsNginx,
            '/',
            'default.conf.template',
            __dirname + '/nginx.default.conf'
          );
        });
    }

[whizz]

Trying to solve the same problem. Interestingly, Azure Classic had support for this (see https://www.pulumi.com/registry/packages/azure/api-docs/storage/sharefile/) but Azure Native does not.