KhronosGroup
Don't reopen redirected standard handles on Windows
Description
If the calling process has redirected a standard handle (e.g. vulkan_samples.exe sample msaa > out.log), then reopening it will detach it from the file/pipe and connect it to the console instead. We don't want to do that.
Instead, the type of the file should be tested before attaching the parent console. If it's already a disk file or a pipe, then we want to leave it alone.
Note that for PowerShell, this doesn't make > file work, but | Out-File file does, and > works in other shells like Command Prompt and MSYS2 Bash. IIRC from past investigation when I've made similar changes to other projects, it's because PowerShell closes the file handle for /SUBSYSTEM:WINDOWS applications immediately.
General Checklist:
Please ensure the following points are checked:
- [x] My code follows the coding style
- [x] I have reviewed file licenses Note: the script wanted to extend the year range of the first line, which I'm not sure is right, as the person it specifies didn't make any changes to this file in 2025.
- [x] I have commented any added functions (in line with Doxygen)
- [x] I have commented any code that could be hard to understand
- [x] My changes do not add any new compiler warnings
- [x] My changes do not add any new validation layer errors or warnings
- [x] I have used existing framework/helper functions where possible
- [x] My changes do not add any regressions
- [ ] I have tested every sample to ensure everything runs correctly I didn't bother with this as it should have the same effect on all samples and doesn't affect Vulkan itself
- [x] This PR describes the scope and expected impact of the changes I am making
Note: The Samples CI runs a number of checks including:
- [x] I have updated the header Copyright to reflect the current year (CI build will fail if Copyright is out of date)
- [ ] My changes build on Windows, Linux, macOS and Android. Otherwise I have documented any exceptions It builds on Windows and is a Windows-specific file, so wouldn't be expected to build elsewhere
If this PR contains framework changes:
- [x] I did a full batch run using the
batchcommand line argument to make sure all samples still work properly
Sample Checklist
N/A
If your PR contains a new or modified sample, these further checks must be carried out in addition to the General Checklist:
- [ ] I have tested the sample on at least one compliant Vulkan implementation
- [ ] If the sample is vendor-specific, I have tagged it appropriately
- [ ] I have stated on what implementation the sample has been tested so that others can test on different implementations and platforms
- [ ] Any dependent assets have been merged and published in downstream modules
- [ ] For new samples, I have added a paragraph with a summary to the appropriate chapter in the readme of the folder that the sample belongs to e.g. api samples readme
- [ ] For new samples, I have added a tutorial README.md file to guide users through what they need to know to implement code using this feature. For example, see conditional_rendering
- [ ] For new samples, I have added a link to the Antora navigation so that the sample will be listed at the Vulkan documentation site
Did try this on windows, but neither Powershell/Terminal nor CMD work as expected. On CMD it breaks things as it seems to be stuck in output, even when returning to the CMD prompt. Powershell with Out-File is kinda works, but leaves you with an empty prompt where you have to press a key for it to exit and actually write to the file.
IMO we need to completely rework this part of our framework. It never worked well, even when not outputting to a file.
Because it's a /SUBSYSTEM:WINDOWS application, shells emit the prompt for the next command immediately, before the application's had a chance to print anything. You should still be able to type the next command and press enter to run it while the Vulkan sample is still running or once it's finished. It's not great, but that's just how Windows works unless you pass the /SUBSYSTEM:CONSOLE flag to the linker, in which case it opens a console when the application starts up, with no way to opt out.
If I had to guess, maybe printing an end-of-file character when the application exits would resolve the problem with PowerShell, although I don't remember that being something that happened in my testing, so it could be a difference between Windows PowerShell and PowerShell Core.
Hi @AnyOldName3, the team decided not to take this change at this time. We may rework this part of the framework in the future to clean this up. LMK if you strongly disagree. Thanks
If the bug isn't being fixed in the immediate future (and this, something very similar, or switching the samples to /SUBSYSTEM:CONSOLE, are the only ways to fix it), it at least should be added to the issue tracker as a bug - I didn't open a report as it was easier to fix than explain.