SCons
Combination of memo stats and substing linker command line fails
In a somewhat obscure combination, if memo stats printing is enabled (--debug=memoizer) and a shared library is created, scons will fail with an exception subst'ing _LIBFLAGS. The variable being expanded that causes the problem is set in the link tool:
env['_LIBFLAGS'] = '${_stripixes(LIBLINKPREFIX, LIBS, LIBLINKSUFFIX, LIBPREFIXES, LIBSUFFIXES, __env__, LIBLITERALPREFIX)}'
Suspicious the LIBLITERALPREFIX construction variable is involved, as that's a relatively new setting appearing in this line, but it might also be due to the shared-library-version symlink handling - that has also received work in recent memory.
A rather obscure error message is emitted, without a backtrace, so you need to add stacktrace to the debug options - and make sure #4677 is applied, as otherwise the memoizer debug won't actually be enabled.
In the failure, _stripixes will eventually create a PathList object; PathList is actually a method of the intentionally obscured _PathListCache, and as a caching dictionary, it is decorated with @SCons.Memoize.CountDictCall. The two memoizing counters are conditional decorators, they are do-nothing unless --debug=memoizer is used to enable them. Somehow the dictionary decorator on PathList changes the call method so in this one particular case too many arguments are passed to the function generating the cache key, leading to this failure:
scons: *** [prog] TypeError `PathListCache._PathList_key() takes 2 positional arguments but 3 were given' trying to evaluate `${_stripixes(LIBLINKPREFIX, LIBS, LIBLINKSUFFIX, LIBPREFIXES, LIBSUFFIXES, __env__, LIBLITERALPREFIX)}'
A reproducer is to "preserve" the output of this existing test:
$ PRESERVE=1 ./runtest.py test/Libs/LIBLITERALPREFIX.py
In the preserved directory, build as:
$ scons --debug=presub,memoizer,stacktrace
This is the slightly trimmed output (I additionally added presub to the debug options to get a bit more verbosity):
$ scons --debug=memoizer,stacktrace
,,,
scons: Building targets ...
Building foo.o with action:
$CC -o $TARGET -c $CFLAGS $CCFLAGS $_CCCOMCOM $SOURCES
gcc -o foo.o -c foo.c
Building libfoo.a with action:
$AR $ARFLAGS $TARGET $SOURCES
ar rc libfoo.a foo.o
Building libfoo.a with action:
$RANLIB $RANLIBFLAGS $TARGET
ranlib libfoo.a
Building shfoo.os with action:
$SHCC -o $TARGET -c $SHCFLAGS $SHCCFLAGS $_CCCOMCOM $SOURCES
gcc -o shfoo.os -c -fPIC shfoo.c
Building libfoo.so with action:
SharedFlagChecker(target, source, env)
Building libfoo.so with action:
$SHLINK -o $TARGET $SHLINKFLAGS $__SHLIBVERSIONFLAGS $__RPATH $SOURCES $_LIBDIRFLAGS $_LIBFLAGS
gcc -o libfoo.so -shared shfoo.os -L.
Building libfoo.so with action:
LibSymlinksActionFunction(target, source, env)
scons: *** [prog] TypeError `PathListCache._PathList_key() takes 2 positional arguments but 3 were given' trying to evaluate `${_stripixes(LIBLINKPREFIX, LIBS, LIBLINKSUFFIX, LIBPREFIXES, LIBSUFFIXES, __env__, LIBLITERALPREFIX)}'
scons: internal stack trace:
File "/home/mats/github/scons/SCons/Taskmaster/Job.py", line 670, in _work
task.prepare()
File "/home/mats/github/scons/SCons/Script/Main.py", line 209, in prepare
return SCons.Taskmaster.OutOfDateTask.prepare(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Taskmaster/__init__.py", line 176, in prepare
self.exception_raise()
File "/home/mats/github/scons/SCons/Taskmaster/__init__.py", line 554, in _exception_raise
raise exc_value.with_traceback(exc_traceback)
File "/home/mats/github/scons/SCons/Taskmaster/__init__.py", line 1035, in next_task
task.make_ready()
File "/home/mats/github/scons/SCons/Script/Main.py", line 356, in make_ready
SCons.Taskmaster.OutOfDateTask.make_ready(self)
File "/home/mats/github/scons/SCons/Taskmaster/__init__.py", line 404, in make_ready_current
t.disambiguate().make_ready()
File "/home/mats/github/scons/SCons/Node/FS.py", line 3133, in make_ready
self.get_binfo()
File "/home/mats/github/scons/SCons/Node/__init__.py", line 1189, in get_binfo
binfo.bactsig = hash_signature(executor.get_contents())
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Memoize.py", line 207, in wrapper
return fn(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Executor.py", line 461, in get_contents
result = bytearray("",'utf-8').join([action.get_contents(all_targets,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Action.py", line 558, in get_contents
result = self.get_presig(target, source, env)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Action.py", line 1346, in get_presig
return c.get_presig(self, target, source, env)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Action.py", line 1117, in get_presig
return env.subst_target_source(cmd, SUBST_SIG, target, source)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Environment.py", line 720, in subst
return SCons.Subst.scons_subst(string, self, raw, target, source, gvars, lvars, conv, overrides=overrides)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Subst.py", line 854, in scons_subst
result = ss.substitute(strSubst, lvars)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Subst.py", line 459, in substitute
result = _dollar_exps.sub(sub_match, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Subst.py", line 454, in sub_match
return self.conv(self.expand(match.group(1), lvars))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Subst.py", line 415, in expand
return self.substitute(s, lv)
^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Subst.py", line 459, in substitute
result = _dollar_exps.sub(sub_match, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Subst.py", line 454, in sub_match
return self.conv(self.expand(match.group(1), lvars))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Subst.py", line 415, in expand
return self.substitute(s, lv)
^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Subst.py", line 459, in substitute
result = _dollar_exps.sub(sub_match, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Subst.py", line 454, in sub_match
return self.conv(self.expand(match.group(1), lvars))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mats/github/scons/SCons/Subst.py", line 393, in expand
raise_exception(e, lvars['TARGETS'], old_s)
File "/home/mats/github/scons/SCons/Subst.py", line 54, in raise_exception
raise SCons.Errors.BuildError(target[0], msg)
scons: building terminated because of errors.
Memoizer (memory cache) hits and misses:
...
