Cannot create DataLake Gen2 container using AZ CLI yet

[pangiole]

This is autogenerated. Please review and update as needed.

Describe the bug

Command Name az storage container create

Errors:

Blob API is not yet supported for hierarchical namespace accounts. ErrorCode: BlobApiNotYetSupportedForHierarchicalNamespaceAccounts
<?xml version="1.0" encoding="utf-8"?><Error><Code>BlobApiNotYetSupportedForHierarchicalNamespaceAccounts</Code><Message>Blob API is not yet supported for hierarchical namespace accounts.
RequestId:9e747912-c01e-00e3-5cb3-776430000000
Time:2019-09-30T17:25:00.0406002Z</Message></Error>
Traceback (most recent call last):
python3.7/site-packages/knack/cli.py, ln 206, in invoke
    cmd_result = self.invocation.execute(args)
...
RequestId:9e747912-c01e-00e3-5cb3-776430000000
Time:2019-09-30T17:25:00.0406002Z</Message></Error>

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

• Put any pre-requisite steps here...
• az storage container create --name {} --account-name {} --account-key {} --auth-mode {}

Expected Behavior

Environment Summary

Darwin-17.7.0-x86_64-i386-64bit
Python 3.7.4
Shell: bash

azure-cli 2.0.74

Extensions:
storage-preview 0.2.8

Additional Context

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @sumantmehtams

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @sumantmehtams

[Juliehzl]

Hi @angiolep, thanks for your reaching out. We already have planned for it with service team. We will do it as soon as possible.

[Riccardo-Fiorentino]

I have a similar issue when using ARM templates { "error": { "code": "ContainerOperationFailure", "message": "Specified feature is not yet supported for hierarchical namespace accounts.\nRequestId:fcce3e69-301e-001e-5cef-b1cc8b000000\nTime:2020-11-03T14:43:26.0342864Z" } }

To recreate just try create a container through an ARM template for a storage account that has Hierarchical enabled.

Any update as to when this will be enabled, I see it was reported over 12 months ago.

[Juliehzl]

@Riccardo-Fiorentino May I know your azure CLI version? It is supported in azure cli already. But for ARM template, add @zhoxing-ms for help. @angiolep CLI command is az storage fs create with azure cli >= 2.6.0.

[Riccardo-Fiorentino]

@Riccardo-Fiorentino May I know your azure CLI version? It is supported in azure cli already. But for ARM template, add @zhoxing-ms for help. @angiolep CLI command is az storage fs create with azure cli >= 2.6.0.

Good day Juliezl, not using cli. Tried using an older API for the ARM, but not functioning.

[zhoxing-ms]

It is a known issue of service that some features are not supported with Azure Data Lake Storage Gen2. As for Blob REST APIs, doc link:

These Blob REST APIs aren't supported:

• Put Blob (Page)
• Put Page
• Get Page Ranges
• Incremental Copy Blob
• Put Page from URL
• Append Block from URL

[zhoxing-ms]

For the error of ARM template, it should also be caused by some features not supported by hierarchical namespace accounts. This requires service team to confirm what and why it is not supported~

{
"error": {
"code": "ContainerOperationFailure",
"message": "Specified feature is not yet supported for hierarchical namespace accounts.\nRequestId:fcce3e69-301e-001e-5cef-b1cc8b000000\nTime:2020-11-03T14:43:26.0342864Z"
}
}

RequestId: 9e747912-c01e-00e3-5cb3-776430000000

@Riccardo-Fiorentino If it's convenient, you can send the ARM template to me by email, my email address is [email protected]

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @armleads-azure.

[SvenAeltermanInfo]

@armleads-azure Is there an update?

Essentially, when exporting an ARM template of an ADLS Gen 2 storage account, you can't re-apply as exported. Only by removing the properties property entirely does the template deploy.

It seems the error might be related to the defaultEncryptionScope property of properties. When it's the only property present, the error changes to this:

Container encryption policy missing header: x-ms-default-encryption-scope and x-ms-deny-encryption-scope-override are required.

When specifying both defaultEncryptionScope and defaultEncryptionScopeOverride, it still fails to deploy.

[mwc360]

I confirmed the same behavior as @SvenAeltermanInfo . The export template cannot be deployed as is, both defaultEncryptionScope and defaultEncryptionScopeOverride need to be removed from the container properties and then the template will deploy.

Very frustrating, it took me about 20 failed test deployments to figure this out. Bicep syntax highlighter even shows it's correct when it is not.

[bqstony]

I had to remove the following two properties in the properies element to work.

"properties": {
   "defaultEncryptionScope": "$account-encryption-key",
   "denyEncryptionScopeOverride": false,
}

now i have access issues

[azurevivekgarg]

Hi Team,

Even after removing those 2 properties from the template.json, there is an undisclosed/unknown error I have encountered within the release pipeline.

Any help would be appriciated.

Thanks Vivek Garg

[mkulisic]

I am encountering the exact same issues. Are there any updates on this?

[evelyn-ys]

Customer can create container(file system) in Data Lake Gen2 storage account using az storage fs create command

[zhoxing-ms]

The problem of ARM deployment still exists. The error information returned by the service is as follows:

{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"Conflict","message":"{\r\n  \"error\": {\r\n    \"code\": \"ContainerOperationFailure\",\r\n    \"message\": \"Specified feature is not yet supported for hierarchical namespace accounts.\\nRequestId:9bd4b751-301e-0046-3900-8651e7000000\\nTime:2022-06-22T06:20:10.4386498Z\"\r\n  }\r\n}"}]}}

'x-ms-correlation-request-id': '1aad2908-4970-4723-b1da-8d31eb0e35ef'

Specified feature is not yet supported for hierarchical namespace accounts

@alex-frankel May I ask do you know who in the ARM service team can help us investigate and clarify this issue?

[TheoDoreW]

Hi, there

I use azcopy command line copy file from Azure blob to Azure Datalake Gen2, also meet with the following error:

409 : 409 Specified feature is not yet supported for hierarchical namespace accounts.. When Creating blob. X-Ms-Request-Id: f9c8b27b-d01e-006c-3c8e-ac78b6000000

both with aad auth and sas key.

Anyone know how to solve this problem? Thanks in advance.

[evelyn-ys]

Hi, there

I use azcopy command line copy file from Azure blob to Azure Datalake Gen2, also meet with the following error:

409 : 409 Specified feature is not yet supported for hierarchical namespace accounts.. When Creating blob. X-Ms-Request-Id: f9c8b27b-d01e-006c-3c8e-ac78b6000000

both with aad auth and sas key.

Anyone know how to solve this problem? Thanks in advance.

Please create an issue for https://github.com/Azure/azure-storage-azcopy repo

[zhoxing-ms]

CLI command is az storage fs create with azure cli >= 2.6.0. Customer can create container(file system) in Data Lake Gen2 storage account using az storage fs create command

Since there is no problem with storage command az storage fs create at present, so the second issue should be a problem of ARM deployment. I guess this may be a problem when the ARM service invokes Storage RP. @wwendyc Could you please help have look at this problem? https://github.com/Azure/azure-cli/issues/10700#issuecomment-1162698536

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @sumantmehtams.

Issue Details

---

This is autogenerated. Please review and update as needed.

Describe the bug

Command Name az storage container create

Errors:

Blob API is not yet supported for hierarchical namespace accounts. ErrorCode: BlobApiNotYetSupportedForHierarchicalNamespaceAccounts
<?xml version="1.0" encoding="utf-8"?><Error><Code>BlobApiNotYetSupportedForHierarchicalNamespaceAccounts</Code><Message>Blob API is not yet supported for hierarchical namespace accounts.
RequestId:9e747912-c01e-00e3-5cb3-776430000000
Time:2019-09-30T17:25:00.0406002Z</Message></Error>
Traceback (most recent call last):
python3.7/site-packages/knack/cli.py, ln 206, in invoke
    cmd_result = self.invocation.execute(args)
...
RequestId:9e747912-c01e-00e3-5cb3-776430000000
Time:2019-09-30T17:25:00.0406002Z</Message></Error>

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

• Put any pre-requisite steps here...
• az storage container create --name {} --account-name {} --account-key {} --auth-mode {}

Expected Behavior

Environment Summary

Darwin-17.7.0-x86_64-i386-64bit
Python 3.7.4
Shell: bash

azure-cli 2.0.74

Extensions:
storage-preview 0.2.8

Additional Context

Author:	pangiole
Assignees:	evelyn-ys, zhoxing-ms
Labels:	Service Attention, ARM, Data Lake, Data Lake Storage Gen2, CXP Attention
Milestone:	-

[navba-MSFT]

Apologies for the late reply. This article talks about the Blob Storage features and describes whether a feature is supported in a standard general-purpose v2 account when you enable a hierarchical namespace (HNS) (ADLS Gen 2).

You can upgrade Azure Blob Storage with Azure Data Lake Storage Gen2 capabilities by following this article. An upgrade is one-way. There's no way to revert your account once you've performed the upgrade. We recommend that you validate your upgrade in a nonproduction environment. More Info here.

@TheoDoreW Do you still need any assistance on this issue ? If Yes, please share the failing X-Ms-Request-Id.

@mkulisic Do you need any assistance on this issue ? If Yes, please share the failing X-Ms-Request-Id.

[ghost]

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!