Karandra
Impossible to run a program from the same process where the VFS is running
The problem
It's not possible to run any program from in the same process where the VFS is running. It doesn't matter what kind of process creation is used (CreateProcess, ShellExecute, etc). An attempt to do so causes VFS unmounting. In case of CreateProcess the functions fails with ERROR_INVALID_PARAMETER.
Environment
- Version: any version.
- Operation system: Windows 7 x64 (should occur on any system)
Details
This is somehow related to security functions KxVFSBase::OnGetFileSecurity and KxVFSBase::OnSetFileSecurity. If they are implemented as in Dokany mirror sample it's not possible to run any process at all no matter in which process it's created. If these functions returns STATUS_NOT_IMPLEMENTED then process creation is possible, but only from another process.
Related issue in Dokany repository: dokan-dev/dokany#650
Workaround
Use proxy process to run VFS and some kind of interprocess communication (IPC) to control VFS.
It seems that latest update (starting with bb32fffe4d37c11476834d06a3d86841ba984d85) solved this issue at least in test console program that builds in debug mode. I'm still unable to run any program in Kortex Mod Manager using same run parameters. Maybe some deadlock happens?