Windows failed to format the volume as NTFS/exFAT/ReFS.

[MarekKnapek]

Steps to reproduce

1. Install VeraCrypt 1.25.9 from 2022-02-19
2. Main menu -> Volumes -> Create New Volume
3. Create encrypted file container
4. Standard VeraCrypt volume
5. select some file, such as c:\dev\disk.vc
6. keep defaults (AES and SHA-512)
7. select some size, such as 1 GB
8. select some password, such as "a", no key file, no PIM
9. In reality I want large container to hold single large file, this file would be a .vhd disk image. So I want primitive file system with large cluster size.
10. Select exFAT and 64k cluster size. Select Quick Format.
11. Click Format button.
12. Progress bar goes to 100 %. UAC prompt appears, select yes.

Expected behavior

New container is created, containing volume formatted as exFAT 64k.

Actual behavior

Error appears, failed to format volume.

---------------------------
VeraCrypt Volume Creation Wizard
---------------------------
Windows failed to format the volume as NTFS/exFAT/ReFS.



Do you want to format the volume as FAT instead?
---------------------------
Yes   No   
---------------------------

---------------------------
VeraCrypt Volume Creation Wizard
---------------------------
Failed to create volume c:\dev\disk.vc
---------------------------
OK   
---------------------------

Work-around

1. This happens also with NTFS and ReFS, but not with FAT. But I cannot use FAT because I want to store large file inside the container/volume.
2. Select none as filesystem. Formatting succeeds.
3. Goto main window and mount your new container to a drive letter.
4. Open command prompt as admin.
5. Execute the format command with some parameters.
6. The format command succeeds to format the volume.

The problem

• VeraCrypt failed to format newly created volume.
• VeraCrypt did not tell me what command (if any) it tried to use to format newly created volume.
• VeraCrypt did not tell me any error code, such as command %errorlevel% nor GetLastError and name of WinAPI function nor NTSTATUS nor HRESULT no-nothing.
• If you don't fix this issue, please at least improve the error message.

Windows Server 2022 Standard, 21H1, version 10.0.20348.1787, amd64.

[idrassi]

Thank you for this report. Normally, you should not see a UAC prompt because it is used only as a fallback when calling the format.com fails. I have never encountered a failure of format.com command on the systems I tested but I have not done tests under Windows Server 2022.

When format.com fails, we fallback to calling an documented function FormatEx exported by the dll fmifs.dll. This function requires administrative privileges and so we run it in a separate process using UAC prompt. The communication between main process and this UAC process is done through a COM interface and currently this interface doesn't implement reporting back the error code of the failure.

I have done some tests on the FormatEx function and it seems to fail when we specify Quick Format. It will succeed only we ask it to perform Full Format.

From here, there 3 things to do:

1. Understand why format.com command executed by VeraCrypt Format fails in your machine.
2. Avoid using undocumented FormatEx function if the user chooses Quick Format and instead report the error encountered by format.com command.
3. Modify the COM interface to add a way to get the error code of FormatEx function (error code is returned by a callback and not the function directly).

Meanwile, can you please help in understanding why format.com fails in your machine by doing the following:

• Create a 1GB file container with no filesystem
• mount it in VeraCrypt as removable media by checking the option in "Mount Option" dialog (this is important)
• open a standard command prompt (not elevated) and type the following command: C:\Windows\System32\format.com Z: /FS:EXFAT /Q /X /V:"" /A:64K
  • Replace Z: in the above command by the drive letter you used for mounting.

what is the error you have? This is exactly how VeraCrypt Format makes use for format.com.

Thank you.

[idrassi]

After reviewing the code responsible for this part, I found two ways to enhance it and make it more robust:

• Using /Y switch in format.com command: The usage of /Y parameter in format.com simplifies the logic and makes the code robust since we don't need to send \n to the format.com process which was making handling of all cases complex and error prone.
• specifying removal media type in FormatEx: pecifying RemovableMedia parameter to FormatEx fixes its failure in Windows 10 and later to perform quick format. This is also more adequate since we are mounting the volume as removable media for the formatting process. FormatEx function is undocumented and so Microsoft do changes to its behavior without prior notice.

Moreover, I added better error management in order to display adequate error messages to the user in case of failure

Here is the commit with this changes: https://github.com/veracrypt/VeraCrypt/commit/0bfed6553d08131fd8e0bd725642ff8509a2a1d4

With this change, you should not have an issue but real test under Windows Server 2022 is needed to confirm. I'm planning to prepare a build today so it would be good if you can test it.

Thank you.

[MarekKnapek]

I did as you said.

Non-admin result:

>format.com Z: /FS:EXFAT /Q /X /V:"" /A:64K
Insert new disk for drive Z:
and press ENTER when ready...
Access Denied as you do not have sufficient privileges or
the disk may be locked by another process.
You have to invoke this utility running in elevated mode
and make sure the disk is unlocked.
Cannot open volume for direct access.
>echo %errorlevel%
4

Admin result:

>format.com Z: /FS:EXFAT /Q /X /V:"" /A:64K
Insert new disk for drive Z:
and press ENTER when ready...
The type of the file system is RAW.
The new file system is EXFAT.
QuickFormatting 1023.8 MB
Initializing the File Allocation Table (FAT)...
Creating file system structures.
Format complete.
    1023.6 MB total disk space.
    1023.4 MB are available.

       65'536 bytes in each allocation unit.
       16'375 allocation units available on disk.

           32 bits in each FAT entry.

Volume Serial Number is D074-BD47
>echo %errorlevel%
0

[idrassi]

Thank you for the test result. It is surprising that format.com fails with Access Denied error. Normally, it should succeed since the volume is mounted as removable media. Maybe a process is accessing the mounted volume as the message says. Anyway, with the new change, the fallback to FormatEx should work since it is run as admin and it uses the correct removalMedia type. A there will be error message displayed with error code in case of failure. As I said, I will create a new build and it would be helpful if you could test it.

[MarekKnapek]

I did some digging why format.com fails. It happens that it calls the ?Initialize@DP_DRIVE@@QEAAEPEBVWSTRING@@PEAVMESSAGE@@EE@Z aka public: unsigned char __cdecl DP_DRIVE::Initialize(class WSTRING const * __ptr64,class MESSAGE * __ptr64,unsigned char,unsigned char) __ptr64 function from IfsUtil.dll module. This function ultimately calls the NtOpenFile function from ntdll.dll module. It does it with DesiredAccess FILE_READ_DATA/FILE_LIST_DIRECTORY | FILE_WRITE_DATA/FILE_ADD_FILE | SYNCHRONIZE, ObjectAttributes.ObjectName \??\VOLUME{6A17DD60-1604-11EE-A6BE-C78B5964F66D}, ShareAccess FILE_SHARE_READ | FILE_SHARE_WRITE, OpenOptions FILE_SYNCHRONOUS_IO_NONALERT, and it fails with NTSTATUS STATUS_ACCESS_DENIED.

[MarekKnapek]

Hello @idrassi, I did some more digging. And I figured out that the formatting fails because Windows (for whatever reason) refuses format.com access to the volume. Administrators (after confirming UAC prompt) have access, non-admin users (or administrators forfeiting the UAC) do not have access. I remembered, that access control is done via ACLs. I searched the Internet for a tool allowing me to display Windows Object Manager's objects with their DACLs. I found the https://github.com/hfiref0x/WinObjEx64 tool made by https://github.com/hfiref0x. I tried to change DACL of already mounted volume, and the tool succeeded to do that (admin not needed). After studying WinAPI documentation and a lot of head-scratching I came up with a way how to change the DACL of the temporary mounted volume from C code. Formatting the volume then succeeds. But there is still one slight problem, changing DACL works only if the volume is mounted as non-removable. If the temporary mounted volume is removable, the DACL cannot be changed and I have no idea why. So I also changed the temporary mounting to be non-removable. This fixes my issue, but I'm not sure whether it is fine to be upstreamed and thus affecting all other users as I'm the only one observing this problem. My change is available on my branch at https://github.com/MarekKnapek/VeraCrypt/tree/issue1138, it is C code, not C++. Best regards, Marek.

[idrassi]

Impressive investigation, @MarekKnapek! Your dedication and analytical skills are truly remarkable.

Concerning the integration of your changes into the main VeraCrypt codebase, there are a couple of potential issues I'd like to highlight:

• Mounting as non-removable: VeraCrypt primarily mounts volumes with the removable flag to ensure compatibility with the FormatEx function. Additionally, this prevents potential issues related to the Windows I/O cache. Changing this default behavior may introduce unforeseen problems.
• Changing the DACL to grant access to everyone: This approach seems overly broad and potentially risky. Additionally, it's unclear if these settings will persist since the volume is mounted as non-removable. If we were to adopt this approach, the code should be modified to revert the DACL back to its original state after formatting is complete to ensure that no unnecessary permissions are left in place.

As I mentioned earlier, I've already implemented a change that should resolve your issue. This involves falling back to a FormatEx API call (which triggers a UAC prompt) if formatting fails. Regrettably, I realized that I didn't share the link to access the Beta version that includes this change. You can find it in the Sourceforge Nightly Builds folder.

I truly appreciate your efforts in analyzing this issue. If you're able, please try out the beta version and let me know how it works for you.

[beefydog]

For others looking for a quick workaround for this: https://sourceforge.net/p/veracrypt/discussion/technical/thread/92d09f967f/

[jgonyea]

I still encounter this bug on: OS: Windows Server 2022 VeraCrypt Version: 1.26.7, Released Oct 1, 2023

FAT formatting completes, but attempting NTFS fails. Attempting to use FAT formatting during the VeraCrypt volume creation and then switching using Windows tools to format the drive also fails.

I've also tried to run VeraCrypt as admin, to the same effect.

My workaround has been to format the drive I want as NTFS first and then encrypt in place.

[idrassi]

@jgonyea Your issue is worse than the one reported by @MarekKnapek since even admin privileges are not enough to allow formatting with NTFS. I guess that using "format.com" in an elevate command prompt also fails. What error do you have when invoking "format.com Z: /FS:NTFS"? This will help understand the nature of the failure. Thank you.

[itkfm]

Ran into an issue similar to jgonyea’s. format.com would complain about partition size vs alignment or something. Unfortunately, I no longer have the exact error message around.

[idrassi]

@itkfm: I've resolved an issue where format.com was reporting errors about the partition offset not being 4K aligned, as documented in this commit. This issue primarily occurred while attempting to encrypt a mounted VHDX volume.

A beta version, 1.26.10, including this fix, has been released and is now available in the Sourceforge Nightly Builds folder. I would greatly appreciate it if you could test this version and provide any feedback you might have. Thank you for your help.

[countzero]

@idrassi I can confirm, that VeraCrypt v1.26.10 fixed the issue in Windows 10:

[leubgi]

Same problem here (Windows 11 with Storage Spaces) Running 1.26.10 x64 Format command on blank filesystem doesn't work either : "Format failed."

[leubgi]

Any update on this ?

[itkfm]

I would greatly appreciate it if you could test this version and provide any feedback you might have.

I worked around the issue by creating a filesystem first and encrypting that then. Unfortunately, this was a client’s system and I no longer have a suitable/similar environment to test the updated version with.

Thanks for the fix anyway.

[leubgi]

Just did that, not working either : "Incorrect function" when I try to browse the disk. What is going on ??

[Unknown78]

Confirmed same issue.. 1.26.10 works on NTFS but not ReFS.

Will it help to rewrite it with pure Rust from scratch? ...

[leubgi]

1.26.10 works on NTFS but not ReFS.

It doesn't work on NTFS

[Unknown78]

It doesn't work on NTFS

It does in my case, create/mount/dismount 10 GiB standard container just fine.

Same problem here (Windows 11 with Storage Spaces)

I read from above that you did it with windows storage space, maybe it's not supported yet?

[leubgi]

Yes, my theory is veracrypt has problem interfacing with the W11 api of storage spaces. Without storage spaces it's working fine I have been using storage spaces on w10 for years without issues

[Unknown78]

Yes, my theory is veracrypt has problem interfacing with the W11 api of storage spaces.

You should create a new issue in the first place instead of commenting on this. It has different chronologies from the original poster and thus make the developers easier to focus.