em++.ps1 passes arguments differently than em++.bat

[jrxpana]

Version of emscripten/emsdk: emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 4.0.15 (09f52557f0d48b65b8c724853ed8f4e8bf80e669) clang version 22.0.0git (https:/github.com/llvm/llvm-project 3388d40684742e950b3c5d1d2dafe5a40695cfc1) Target: wasm32-unknown-emscripten Thread model: posix

Problem Description In previous versions of emscripten, calling em++ with a specific set of command line arguments succeeded. After we updated to 4.0.15 it suddenly failed. I traced it back to the fact that the previous versions we used had 3 versions of the file/command:

• em++
• em++.bat
• em++.py

and 4.0.15 now also has:

• em++.ps1

When our build script calls em++ it now ends up calling em++.ps1 instead of em++.bat, which somehow passes the arguments to python differently.

Minimal repro:

hi.cpp

#include <iostream>

int main()
{
	std::cout << "hi" << std::endl;
	return 0;
}

build.ps1

$emsdk_path = "$env:LOCALAPPDATA\emsdk"

& "$emsdk_path\emsdk.ps1" construct_env

Write-Host "build using em++.bat" -fore green
& em++.bat -pthread -s ALLOW_MEMORY_GROWTH=1 hi.cpp 2>&1 | % ToString | Tee-Object -Variable build_result

Write-Host "build using em++.ps1" -fore green
& em++.ps1 -pthread -s ALLOW_MEMORY_GROWTH=1 hi.cpp 2>&1 | % ToString | Tee-Object -Variable build_result

With both of these files in the same folder, when I open a Powershell window and run .\build.ps1, I get the following output:

Setting up EMSDK environment (suppress these messages with EMSDK_QUIET=1) Setting environment variables:

build using em++.bat em++: warning: -pthread + ALLOW_MEMORY_GROWTH may run non-wasm code slowly, see https://github.com/WebAssembly/design/issues/1271 [-Wpthreads-mem-growth]

build using em++.ps1 python.exe : em++: warning: -pthread + ALLOW_MEMORY_GROWTH may run non-wasm code slowly, see https://github.com/WebAssembly/design/issues/1271 [-Wpthreads-mem-growth] At C:\Users\yournamehere\AppData\Local\emsdk\upstream\emscripten\em++.ps1:38 char:1 + & $launcher $launcherArgs $pythonScript $MyInvocation.UnboundArgument ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (em++: warning: ...ads-mem-growth]:String) [], RemoteException + FullyQualifiedErrorId : NativeCommandError

The .bat command works, the .ps1 command fails. We are piping the output to a build_result variable so we can examine it later; if we remove the 2>&1 | % ToString | Tee-Object -Variable build_result stuff at the end of the command both commands work fine.

We have another similar issue with emcmake.ps1 / emcmake.bat, but in that case it's because arguments that are sent using Powershell splatting are somehow passed differently than arguments that are passed directly to the command. I can create a separate issue for that.

[sbc100]

Are you able to identify the specific issue with the powershell script?

Perhaps @RReverser has some idea since he added this ps1 scripts back in #20416

[jrxpana]

I've tried poking around at the em++.ps1 file a bit, but unfortunately I haven't been able to find a fix yet. I'm honestly not that skilled with PowerShell.

I did discover that removing just the 2>&1 makes everything work again, at least in this case. So maybe this specific issue has to do with the stderr redirection.

[RReverser]

Hm from a quick search sounds like the same issue as described in https://forums.powershell.org/t/redirect-stderr-using-tee-object-unexpected-behavior/24264/5.

[sbc100]

Maybe another reason to drop the launcher scripts completely. See #24858.

[RReverser]

Yeah that does look increasingly more appealing.

[jrxpana]

Thanks for taking a look.

@RReverser I agree that it's just like the situation in that powershell.org link, but the proposed solution in that case is the same thing we're doing in the reproducer where we pipe to % ToString before we pipe to Tee-Object. That does work if we call PowerShell >> BAT file >> python but somehow it doesn't if we call PowerShell >> PowerShell >> python.

I did some research and it seems that PowerShell treats stderr in some special way that I don't understand, and I could never find a workaround. The only thing that seemed to work was using Start-Process instead of & but then all of the output got lost unless you redirected stdout and strerr to a file and then spit the output back when it's done (apparently you can't redirect to a variable). That's not really a good solution. ProcessStartInfo can keep the streams in memory but still has to spit them back out, and it seems to be prone to hangs.

Anyway, for now as long as you keep the .bat versions of everything we can work around it. If you end up using .exe files instead I imagine it'll be fine too, and if not we'll cross that bridge when we get there.