Enable data deduplication on DevDrive on Windows 11

[cd21h]

Suggested new feature or improvement

Enable ReFS data deduplication feature for DevDrive on Windows 11.

Scenario

As a Developer I want to have an ability to configure data deduplication on DevDrive on Windows 11 So that I can improve my disk performance and storage size.

Additional details

ReFS data deduplication is not available on Win11.

Ideally third-party package managers should support copy-on-write feature. In case support is not available, deduplication should help with optimizing storage

[danieljennings]

You can do this on Windows 11 24H2 if you use refsutil dedup or the ReFS PowerShell commands detailed here: https://learn.microsoft.com/en-us/azure-stack/hci/manage/refs-deduplication-and-compression?tabs=powershell

My V: is a normal dev drive:

>fsutil devdrv query V:
This is a trusted developer volume.

Developer volumes are protected by antivirus filter, by group policy.

Filters currently attached to this developer volume:
    WdFilter

Here's refsutil measuring the deduplication savings on that drive:

>refsutil dedup /s V:
V:\ is an ReFS volume.
Using 4096 byte chunk size
Requested maximum 50% CPU utilization
Creating weak references for all candidate ranges.
All candidates processed.
Estimating space savings on V:\...
Status: 100% complete, 62324 read, 62324 candidates
Estimation complete (duration: 5.99 minutes).

Error counts:
Hash index update errors: 0
Hash index update transaction errors: 0
Maximum Tx retries due to log file full: 0
FSCTL_DUPLICATE_CLUSTER errors: 0
Source has maximum allowed references, no longer dedupable: 0
Source invalid errors: 0
Target invalid errors: 0
Not supported errors: 0
VCN->LCN mapping changed errors: 0

Undedupable data counts:
Stream reserved LCNs: 0
Read only LCNs: 0
Invalid (allocated, but undefined data) LCNs: 0

Processed file ranges: 62324
Bytes deduped during optimization: 0.00 B
Processed data size (logical): 329.64 GiB
Total shared data: 250.14 GiB
Space savings percent (no compression): 76%

And after Enable-ReFSDedup -Volume V: -Type Dedup and completion of Start-ReFSDedupJob -Volume V:, my disk usage has dropped significantly and the volume shows up in logs as "DedupEnabled: true"

[cd21h]

You can do this on Windows 11 24H2 if you use refsutil dedup or the ReFS PowerShell commands detailed here: https://learn.microsoft.com/en-us/azure-stack/hci/manage/refs-deduplication-and-compression?tabs=powershell

Thank you! I was on 23H2, after upgrade to 24H2 it worked! 51% space savings without compression :)

[greenhat616]

Hi. I am facing this error code GLE: 0x80070032 when run refsutil dedup /s G:. The kenerl info: Kernel: WIN32_NT 10.0.26100.2454 (24H2)

Could you share your kenerl ver whether is the same? I hope it just a pre-release issue, and i can reinstall latest iso to fix up it.

[cd21h]

@greenhat616

Hi. I am facing this error code GLE: 0x80070032 when run refsutil dedup /s G:. The kenerl info: Kernel: WIN32_NT 10.0.26100.2454 (24H2)

Could you share your kenerl ver whether is the same? I hope it just a pre-release issue, and i can reinstall latest iso to fix up it.

Got the same error after I ran dedup win version: 10.0.26100 N/A Build 26100

[cocoon]

error code GLE: 0x80070032

Seems that Disable-ReFSDedup G:

makes it compatible with refsutil again?

[cd21h]

error code GLE: 0x80070032

Seems that Disable-ReFSDedup G:

makes it compatible with refsutil again?

Worked for me. Thanks a lot!