Make enabling "Include Directory Stubs" in Settings also add --include-directory-stub flag when uploading for local to blob

[jimbobur]

Problem

I'm uploading a directory tree with empty subfolders in it. Enabling "Include Directory Stubs" in Settings doesn't do anything when uploading for local to blob; under the bonnet AZCopy is not called with --include-directory-stub flag when uploading. I have confirmed this by uploading a folder with Storage Explorer with with empty subfolders and seeing that they have not being uploaded†, and by clicking "Copy AZCopy command to Clipboard" on the upload message in Activities, showing that no --include-directory-stub flag was included.

† Yes I also enabled "Show Stub Blobs for Virtual Directories" in Settings.

When uploading a directory tree with empty subfolders in it using AZCopy directly with the --include-directory-stub flag set, such empty folders are recreated as 0-byte directory stubs with metadata 'hdi_folder=true in the Azure storage location being uploaded to.

Desired Solution

When "Include Directory Stubs" in Settings, pass --include-directory-stub to AzCopy.

Alternatives and Workarounds

Currently the only way to preserve (recreate as stubs) empty subfolders on upload is to use AZCopy directly with a command of the form:

azcopy copy 'D:\path\to\local\az_test_folder' `
'https://my-storage-account.blob.core.windows.net/test-storage-container/' `
--block-blob-tier=Hot `
--from-to=LocalBlob `
--blob-type Detect `
--follow-symlinks `
--check-length=true `
--put-md5 `
--recursive `
--include-directory-stub

Additional Context

I opened this as a feature request instead of a bug as the info text for "Include Directory Stubs" suggests this behaviour is intentional and that directory stubs are only included for service-to-service transfers and not local-to-blob:

Following further context from @craxal , perhaps there could be a separate option checkbox that says something like "Recreate empty folders as stubs for local-to-blob uploads" that toggles on --include-directory-stub to AZCopy for local-to-blob uploads (cf. command in my workaround).

---

Update 2025-11-06: thanks and apologies to @craxal re: the misleading language when I originally wrote this. I meant to refer to uploading a directory tree with empty subfolders, not uploading a directory tree with stubs.

[craxal]

@jimbobur You are correct. The --include-directory-stub option is set based on the "Include Directory Stubs" setting only for service-to-service (such as by copying and pasting blobs).

Why did we do it this way? The description for the settings, as well as azcopy copy --help says that directory stubs are blobs with metadata 'hdi_folder=true'. Local files aren't blobs and don't have blob metadata, so it is not clear how a local file could be interpreted as a directory stub without some additional manipulation.

With your workaround, do you observe directory stub files getting uploaded as directories? Do your directory stub files have special attributes or properties? I assume this is all with a classic blob container and not a container with hierarchical namespaces enabled (ADLS Gen2)?

[jimbobur]

Why did we do it this way? The description for the settings, as well as azcopy copy --help says that directory stubs are blobs with metadata 'hdi_folder=true'. Local files aren't blobs and don't have blob metadata, so it is not clear how a local file could be interpreted as a directory stub without some additional manipulation.

With your workaround, do you observe directory stub files getting uploaded as directories? Do your directory stub files have special attributes or properties? I assume this is all with a classic blob container and not a container with hierarchical namespaces enabled (ADLS Gen2)?

Hi, @craxal ! Yes, passing --include-directory-stub to AZCopy when uploading a directory tree containing empty folders to blob storage results in the creation of corresponding 0-byte stubs with metadata hdi_folder=true in the blob storage location being uploaded to. This is what I'm doing in my use-case. Apologies, the language in my feature request with regard to what I'm uploading was inaccurate/misleading - I should've referred to the empty local folders as such and not called them 'stubs'. I'll edit the post.

[craxal]

Thanks for confirmation. We'll review and see about lifting the service-to-service restriction.

[craxal]

@jimbobur Since you appear to want real folders, have you considered enabling hierarchical namespaces (migrating to ADLS Gen2)? In HNS blob containers, folders are real objects, whereas in classic blob containers, folders don't really exist. One of the reasons we don't use --include-directory-stubs was to avoid creating confusion between classic and HNS enabled blob containers that could lead users to unexpected behavior (stub blobs were sort of a way to cheat having folders until HNS came along).

[jimbobur]

@craxal Our use case is large-scale data backup; the interest in empty folders was just for the sake of preserving directory structure 100% as-is, but we may be fine just living with losing empty dirs (or at the very least creating stub blobs).

In future we may just take the local CPU/memory/time hit and 7zip the data before upload. We may also look into HNS although, if my understanding is correct, for our purposes (data backup, write-once, read-rarely) using HNS would mean an increase in per-op costs for not much benefit (for our particular use case, since we never rename/overwrite backed-up data).

[craxal]

@jimbobur Our team is discussing options. Implementation is not too difficult, but as I said earlier, we don't want users to get confused about the different behaviors between classic and HNS blobs.

You may look over this article when considering costs. If you choose to upgrade, there are other things you can do to minimize costs, such as modifying the access tier of your data.