Azure as Primary Storage

[chaitanyayeleti]

Hello Team ,

Thank you so much for the wonderful product which u people created as Open Source.

from last one year I'm using Nextcloud which is deployed on azure Linux VM . recently i started testing docker image and trying to configure azure blob/file share as primary data storage for the docker image . but there is no documentation about that for standalone installation there is but for docker its is not available in inside docker image config nothing related to azure only aws and open...

can some one help to implement this or any help

[joshtrichards]

Hi @chaitanyayeleti -

I'm surprised this hasn't come up before with the community Docker image, but I can't find any history of it arising.

You have three choices I'd say:

1. Migrating to Azure for Primary Storage after initial installation
2. Use a pre-installation hook script to configure things (I think this would work for that). See https://github.com/nextcloud/docker/#auto-configuration-via-hook-folders
3. Adapt the code for S3 or SWIFT then when you have it working submit it as a pull request so it gets integrated. This would probably be best if you're willing to do a bit more upfront work and some very light coding. See https://github.com/nextcloud/docker/blob/38b8110c58782cc283c3a784c1ddbe8f90f3cec3/.config/s3.config.php for a model to implement support for the OBJECTSTORE_AZURE_* modeled after the current S3 support.

[chaitanyayeleti]

Hi @joshtrichards ,

so i got the code for making azure as primary storage we can choose either blob or fileshare .

For File Share

if (getenv('AZURE_FILE_STORAGE_ACCOUNT_NAME') && getenv('AZURE_FILE_STORAGE_ACCOUNT_KEY')) { $use_ssl = getenv('AZURE_FILE_STORAGE_SSL');

$CONFIG = array(
    'objectstore' => array(
        'class' => '\OC\Files\ObjectStore\AzureFile',
        'arguments' => array(
            'account_name' => getenv('AZURE_FILE_STORAGE_ACCOUNT_NAME'),
            'account_key' => getenv('AZURE_FILE_STORAGE_ACCOUNT_KEY'),
            'share_name' => getenv('AZURE_FILE_SHARE_NAME') ?: 'your-file-share-name',
            'use_ssl' => (strtolower($use_ssl) === 'false' || $use_ssl == false) ? false : true,
        )
    )
);

}

For Blob

if (getenv('AZURE_BLOB_STORAGE_ACCOUNT_NAME') && getenv('AZURE_BLOB_STORAGE_ACCOUNT_KEY')) { $use_ssl = getenv('AZURE_BLOB_STORAGE_SSL'); $use_path = getenv('AZURE_BLOB_STORAGE_USEPATH_STYLE');

$CONFIG = array(
    'objectstore' => array(
        'class' => '\OC\Files\ObjectStore\AzureBlob',
        'arguments' => array(
            'account_name' => getenv('AZURE_BLOB_STORAGE_ACCOUNT_NAME'),
            'account_key' => getenv('AZURE_BLOB_STORAGE_ACCOUNT_KEY'),
            'container' => getenv('AZURE_BLOB_STORAGE_CONTAINER_NAME') ?: 'your-container-name',
            'use_ssl' => (strtolower($use_ssl) === 'false' || $use_ssl == false) ? false : true,
            'use_path_style_uri' => $use_path == true && strtolower($use_path) !== 'false',
        )
    )
);

}

can you help me implement the code i refered the code https://learn.microsoft.com/en-us/previous-versions/azure/storage/blobs/storage-quickstart-blobs-php?tabs=windows

[joshtrichards]

That's great!

Can you submit it as a PR?

[kevinmccurdybrd]

Hey @joshtrichards I'm new to this issue and see it's sort of slowed in progress. I'm looking to get this PR'ed and in (as a newcomer to PR's please be patient with me, doing my very best to do it right).

I have a question, is it desirable to construct this as a single config file (azure.config.php) for OBJECTSTORE_AZURE and use if statements to separate it down to either AzureFiles or AzureBlob (as @chaitanyayeleti has written it), or does it make more sense (and how I read your latest point at this item based on your "OBJECTSTORE_AZURE_*" comment) that there be two unique config files: (azureblob.config.php and azurefiles.config.php) for OBJECTSTORE_AZURE_BLOB, and OBJECTSTORE_AZURE_FILES respectively?

The nextcloud docs suggest the class needs to be "\OC\Files\ObjectStore\Azure" per [here] (https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/primary_storage.html#microsoft-azure-blob-storage) so I'm wondering if that suggests the former is required.

I think I can do it either way, just looking for insight as to how the NC team would prefer it to be implemented. Apologies if this is a stupid question.

I could use this functionality and want to PR it in ASAP. Thanks for your time on this, and appreciate your patience with me.

[joshtrichards]

Looking back at what @chaitanyayeleti posted, I think including Files alongside Blob was an oversight.

Azure Blob is AFAIK the only thing in scope here unless I'm missing something. That's the Azure Object Storage solution. And it's what is supported for usage as Primary Storage within Nextcloud. So, yeah, the class needs to be the \OC\Files\ObjectStore\Azure like in the docs.

Azure Files can be integrated, but I don't see the Docker images needing anything special for that. It's just exposed via NFS or SMB it looks like from Microsoft's docs. So that's a different situation entirely (i.e. not related to object storage support that is built into Nextcloud).

So I think the answers to what you're getting at is... you can probably, well, I'd vote for:

• azure.config.php being sufficient
• OBJECTSTORE_AZURE_KEY (etc etc) being sufficient

Closely modeling after the existing S3 support in terms of auto-config variable naming (as much as possible/makes sense) to maintain consistency.

One note, to merge this we'll need to support Docker secrets too. Fortunately that's pretty easy and can just be replicated in the same way as it exists for S3 via the _FILE references in s3.config.php for the sensitive variables.

P.S. Well I guess we should call it blob.config.php for consistency, but my justification (personally) for not pushing that is that Azure Blob (unlike S3) isn't used by anybody else but Azure. And "Azure" seems more clear to me. :man_shrugging: What do you think?

[joshtrichards]

Oh, and thanks for following up on this! :)

[kevinmccurdybrd]

<?php
if (getenv('OBJECTSTORE_BLOB_ACCOUNT_NAME')) {
  $use_ssl = getenv('OBJECTSTORE_BLOB_SSL');
  $use_path = getenv('OBJECTSTORE_BLOB_USEPATH_STYLE');
  $CONFIG = array(
    'objectstore' => array(
      'class' => '\OC\Files\ObjectStore\Azure',
      'arguments' => array(
        'account_name' => getenv('OBJECTSTORE_BLOB_ACCOUNT_NAME'),
        'account_key' => getenv('OBJECTSTORE_BLOB_KEY'),
        'container' => getenv('OBJECTSTORE_BLOB_CONTAINER_NAME') ?: 'your-container-name',
        'use_ssl' => (strtolower($use_ssl) === 'false' || $use_ssl == false) ? false : true,
        'use_path_style' => $use_path == true && strtolower($use_path) !== 'false',
      )
    )
  );

  if (getenv('OBJECTSTORE_BLOB_KEY_FILE') && file_exists(getenv('OBJECTSTORE_BLOB_KEY_FILE'))) {
    $CONFIG['objectstore']['arguments']['key'] = trim(file_get_contents(getenv('OBJECTSTORE_BLOB_KEY_FILE')));
  } elseif (getenv('OBJECTSTORE_BLOB_KEY')) {
    $CONFIG['objectstore']['arguments']['key'] = getenv('OBJECTSTORE_BLOB_KEY');
  } else {
    $CONFIG['objectstore']['arguments']['key'] = '';
  }

  if (getenv('OBJECTSTORE_BLOB_SECRET_FILE') && file_exists(getenv('OBJECTSTORE_BLOB_SECRET_FILE'))) {
    $CONFIG['objectstore']['arguments']['secret'] = trim(file_get_contents(getenv('OBJECTSTORE_BLOB_SECRET_FILE')));
  } elseif (getenv('OBJECTSTORE_BLOB_SECRET')) {
    $CONFIG['objectstore']['arguments']['secret'] = getenv('OBJECTSTORE_BLOB_SECRET');
  } else {
    $CONFIG['objectstore']['arguments']['secret'] = '';
  }
} 

Something like this? Just looking for a quick review before I turn it into a PR.

WRT your last note, personally I'm screaming Azure, but when I look at it from a naming convention side of things OBJECTSTORE_BLOB_ACCOUNT_NAME seems better than OBJECTSTORE_AZURE_ACCOUNT_NAME, and neither of them do justice to OBJECTSTORE_AZURE_STORAGE_ACCOUNT_NAME which is what it actually is. STORAGE_ACCOUNT only makes sense for some of the variables though.

[joshtrichards]

:+1: Looks in the ballpark from a quick glance. Maybe just double-check all those parameters are valid for blobs. Some of those look S3 specific.

I'm thinking OBJECTSTORE_AZURE_<parameter> is the right approach. Keeps things consistent with the Nextcloud class name's (swift, s3, azure).

[kevinmccurdybrd]

Ok I'll get that sorted out. It's interesting because S3 and SWIFT are both technologies, while Azure is the platform, so it's sort of the odd one out by that standard. I think it probably should have been Blob as a class, but c'est la vie.