SCons
Change CI provisioning
Sphinx unpinned. lxml pinned as <5 rather than a specific version.
Experimentally, add two workflows: Ubuntu 24.04 and Windows 2022
For the Windows workflow run, test test\MSVC\msvc.py fails due to the precompiled headers speed test.
In #4534, I removed the multiplier (i.e., effectively 1.0):
# TODO: Reevaluate if having this part of the test makes sense any longer
# using precompiled headers should be faster
- limit = slow*0.90
+ limit = slow
if fast >= limit:
print("Using precompiled headers was not fast enough:")
print("slow.obj: %.3fs" % slow)
In hindsight, it probably should have been changed to slow*1.0 (or even marginally larger than 1).
This test fails frequently for me when running tests in virtual machines. Even with the change, it still occasionally fails in one vm (i.e., pre-compiled headers slower).
There are some mysterious fails due to adding the Windows runner, I guess I should have listed them here.
Failed the following 39 tests:
test\CPPDEFINES\pkg-config.py
test\Interactive\added-include.py and 25 others (of 28 Interactive total)
test\MSVC\msvc.py
test\packaging\msi\explicit-target.py
test\packaging\msi\file-placement.py
test\packaging\msi\package.py
test\packaging\tar\xz_packaging.py
test\scons-time\run\config\python.py
test\scons-time\run\option\python.py
test\scons-time\run\option\quiet.py
test\scons-time\run\option\verbose.py
test\sconsign\script\no-SConsignFile.py
test\sconsign\script\SConsignFile.py
test\sconsign\script\Signatures.py
Many of these are mystifying. The sconsign tests fail because the third field of entries for files is -1, which doesn't match the regex expressions \d+. But why is it getting -1?
The scons-time test - the python.py ones fail by missing the "command echo" from the files, the other two fail because we get back a path that's gone through 8+3 shortening, like:
< 'SConstruct file directory: C:\\\\Users\\\\runneradmin\\\\AppData\\\\Local\\\\Temp\\\\scons\\-time\\-[^\\\\]*\\\\foo'
---
> 'SConstruct file directory: C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\scons-time-pim4q763\\foo'
The packaging tests fail with an error I've never seen,
scons: *** Missing Packagetag 'X_MSI_LANGUAGE' for SCons.Tool.packaging.msi packager
... even though the packaging tool itself sets that.
The pkg-config test fails because perl is not found, but it's also not used !?!?!? And don't know where it found that path for pkg-config, it should be unable to look in such a weird path.
STDERR =========================================================================
'perl' is not recognized as an internal or external command,
operable program or batch file.
OSError: ' "C:/Strawberry/perl/bin/pkg-config.BAT" --cflags bug.pc' exited 9009:
There are a few others. Not able to spend time chasing these down at the moment.
The pkg-config test fails because perl is not found, but it's also not used !?!?!? And don't know where it found that path for pkg-config, it should be unable to look in such a weird path.
The pkg-config.py test contains
pkg_config_path = test.where_is('pkg-config')
On windows, when using the default pathext argument(i.e., pathext=None) for the where_is function invocation, all path extensions are tested with the basename. It appears that the Strawberry Perl bin folder contains a pkg-config.BAT which must be on the system path.
I could be wrong...
On windows, when using the default
pathextargument(i.e.,pathext=None) for thewhere_isfunction invocation, all path extensions are tested with the basename. It appears that the Strawberry Perl bin folder contains apkg-config.BATwhich must be on the system path.I could be wrong...
Unlikely to be wrong. Yes, the tests are sometimes more ambitious than SCons is going to be with its limited PATH in the execution environment. I still don't know why perl gets involved.
I've pushed a change to this PR which skips the 39 tests that failed last time, which has gotten us a "pass" this time. I'll record those in an issue and we can whack those off (and reduce the skip list) as motivation appears - no rush, really.
The following could be wrong...
I still don't know why perl gets involved.
The where-is call is returning the fully qualified path to the Strawberry Perl batch file pkg-config.bat. The Strawberry Perl batch file pkg-config.bat calls perl.
pkg-config.bat:
@rem = '--*-Perl-*--
@set "ErrorLevel="
@if "%OS%" == "Windows_NT" @goto WinNT
@perl -x -S "%0" %1 %2 %3 %4 %5 %6 %7 %8 %9
@set ErrorLevel=%ErrorLevel%
@goto endofperl
:WinNT
@perl -x -S %0 %*
@set ErrorLevel=%ErrorLevel%
@if NOT "%COMSPEC%" == "%SystemRoot%\system32\cmd.exe" @goto endofperl
@if %ErrorLevel% == 9009 @echo You do not have Perl in your PATH.
@goto endofperl
@rem ';
#!/usr/bin/perl
#line 16
...
If the fully-qualified pkg-config.bat call is made in a restricted SCons environment, perl won't be on the path.
Never heard of Strawberry Perl, so I didn't realize it was a "thing", just thought the CI image had some weird paths. But it is a Thing indeed... http://www.strawberryperl.com/.
So yeah, we found something in the more liberal test framework search, and then the necessary path wasn't in env['ENV']['PATH'] when scons fired off the test. Not entirely sure how to fix that, but there will be a way...