SCons
SCons does not rebuild executables when static libraries dependencies change
This issue was originally created at: 2009-09-21 17:43:42.
This issue was reported by: jcoffland.
jcoffland said at 2009-09-21 17:43:42
Sorry about the comments in 2303 and accidental attachment on 2304. I now think this is a separate issue.
I have been having problems with static library dependencies. A static library built with one scons script and included via -L<PATH> -l<name> in another scons script does not cause executables to be rebuilt. In other words, SCons never seems to rebuild based on changes solely to the static library. Simple example attached.
The attachment SCons-Static_Library_Failed_Change_Detection.tar.bz2 can be used like this:
tar xjvf SCons-Static_Library_Failed_Change_Detection.tar.bz2
cd SCons-Static_Library_Failed_Change_Detection
scons -C lib
scons -C prog
# Make a change to lib/hello.c
scons -C lib
scons -C prog # Does not rebuild!
I've seen this bug on several flavors of Linux and in Windows. Python 2.5 and 2.6. SCons version 1.2.0 and possibly earlier.
Add the following to your SConscript to work around this problem:
class static_lib_decider_hack:
def __init__(self, env):
self.env = env
self.decider = env.decide_source
def __call__(self, dep, target, prev_ni):
if str(dep).endswith(self.env['LIBSUFFIX']):
try:
csig = dep.csig
except AttributeError:
csig = MD5signature(dep.get_contents())
dep.csig = csig
dep.get_ninfo().csig = csig
if prev_ni is None: return True
return csig != prev_ni.csig
return self.decider(dep, target, prev_ni)
env.Decider(static_lib_decider_hack(env))
jcoffland said at 2009-09-21 17:44:13 Created an attachment (id=632)
Simple example of static library change detection bug
jcoffland said at 2009-09-21 17:45:32
This has caused me lots of headaches in an otherwise awesome build tool.
bdbaddog said at 2009-09-21 22:41:27
I ran the testcase and the bug is reproducible.
garyo said at 2009-11-03 16:56:59
This seems to be related to the Configure test. You always should call conf.Finish() after any config test, see the man page. However, even if I add that, it doesn't help. But if I omit the configure test and manually say env.Append(LIBS=['hello']) then all is fine.
gregnoel said at 2009-12-09 11:32:49
Bug party triage. Test case is bogus, so Gary to work with OP to correct problems. Actual problem may be that CPPPATH is missing. If problem is real, reassign to be reconsidered at a future bug party.
jcoffland said at 2009-12-09 12:04:50
Greg: The test case was confirmed by William Deegan and I doubt it is because CPPPATH is not set. For one CPPPATH is for finding includes not libraries. And I am not stupid enough to report such a bug when my LIBPATH is not set correctly. I have looked in to this in some detail and was careful to submit a clear and simple test case.
Sorry to be a bit harsh but rather than assume the bug reporter is ignorant you should look in to it in more detail. If you just blow through bugs and throw them out you are hurting rather than helping the project. If you think my test case is "Bogus" please explain why.
I am a software developer myself and know how hard it is to please users and keep up with bug reports but come on man!
garyo said at 2009-12-09 12:35:26
Hi Joseph; can you try your testcase with --debug=findlibs turned on? (And conf.Finish() added in the appropriate place.) That ought to give more info. Also --tree=prune may show the missing dependencies. But it may go deeper than that since it doesn't happen without the Configure() test.
jcoffland said at 2009-12-09 12:56:04
Thank you for revisiting this.
I added conf.Finish() to both SConstruct files. It did not seem to make any difference. --debug=findlibs also didn't seem to do anything.
Here is the output with those two options. I rebuilt the library right before this with an change to the code (removed one of the printf()s). So the program should rebuild but it does not.
jcoffland@kenny:~/projects/scons/SCons-Static_Library_Failed_Change_Detection/prog$
scons --debug=findlibs
scons: Reading SConscript files ...
Checking for C library hello... (cached) yes
scons: done reading SConscript files.
scons: Building targets ...
scons: `.' is up to date.
scons: done building targets.
jcoffland@kenny:~/projects/scons/SCons-Static_Library_Failed_Change_Detection/prog$
scons --debug=findlibs --tree=prune
scons: Reading SConscript files ...
Checking for C library hello... (cached) yes
scons: done reading SConscript files.
scons: Building targets ...
scons: `.' is up to date.
+-.
+-.sconf_temp
| +-.sconf_temp/conftest_0
| | +-.sconf_temp/conftest_0.o
| | | +-.sconf_temp/conftest_0.c
| | | | +-
int
main() {
return 0;
}
| | | +-/usr/local/bin/gcc
| | +-/usr/local/bin/gcc
| |
+-/home/jcoffland/projects/scons/SCons-Static_Library_Failed_Change_Detection/lib/libhello.a
| +-[.sconf_temp/conftest_0.c]
| +-[.sconf_temp/conftest_0.o]
+-SConstruct
+-config.log
+-hello
| +-main.o
| | +-main.c
| |
+-/home/jcoffland/projects/scons/SCons-Static_Library_Failed_Change_Detection/lib/hello.h
| | +-/usr/local/bin/gcc
| +-/usr/local/bin/gcc
|
+-/home/jcoffland/projects/scons/SCons-Static_Library_Failed_Change_Detection/lib/libhello.a
+-main.c
+-[main.o]
scons: done building targets.
gregnoel said at 2010-07-21 16:58:11
Bug party triage. Bump this issue to P1 so it will be reviewed at next bug party.
garyo said at 2010-08-12 17:04:08
I have more info; if you look at the config.log from the first build, you can see something very weird happening. libhello.a is actually getting built during the Configure() step! If you look at the output of a clean build, you see it never says it's building the lib (but it ends up built). That's because it's getting built during configure. So it's probably not really getting into the scons dependency graph properly somehow. I'll keep poking...
garyo said at 2010-08-23 20:37:18
Still don't have a definitive solution, but I have a pretty good idea what's going on. We're looking at the situation after everything's built and the hello.c (lib source) has been modified, and that lib has been rebuilt by its SConstruct (so we're just about to do the final prog build). Note that there's no connection between the lib's SConstruct and the prog's. That's important; in the prog's SConstruct, the lib is just a source Node. In the prog SConstruct, which looks like this:
env = Environment()
conf = Configure(env)
env.Append(CPPPATH = ['../lib'])
env.Append(LIBPATH = ['../lib'])
conf.CheckLib('hello')
conf.Finish()
env.Program('hello', 'main.c')
the conf.CheckLib line links a temp configure program with libhello.a using SCons. I suspect that is updating the Node for lib/libhello.a to have its updated content signature (this happens while the SConstruct is being read), and then when env.Program is called it compares the csig of that lib on disk with the one in memory (usually the one it read in from the .sconsign file, but here it's been updated by the CheckLib call I think) and they match so it thinks it doesn't need to relink.
In order to prove it I'll have to print out some node infos at various places. Fixing it I'm less sure of; I'm not sure how to isolate any changes the Configure subsystem might make.
gregnoel said at 2010-09-06 12:07:11
Bug party triage.
weegreenblobbie said at 2013-02-26 09:20:07
I just ran into this bug and am surprised how old this ticket is and wondering why it's not priority 1!
One of the main attractions to scons is the auto dependency tracking, but without it, it's not SCons! SCons is broken until this is fixed!
This is a fundamental feature and it's ridiculous this bug hasn't been resolved yet.
Grumble grumble grumble!
weegreenblobbie said at 2013-02-26 09:23:23
Oh yeah, I'm using scons v2.1.0.r5357[MODIFIED], 2011/09/09 21:31:03, by bdeegan on ubuntu.
weegreenblobbie said at 2013-02-26 09:31:56
My HACK / WORKAROUND was to explicitly add headers to the source list:
import os
libfoo_SRC = """
file1.cc
file2.cc
...
""".split()
# Hack / workaround scons bug 2485:
http://scons.tigris.org/issues/show_bug.cgi?id=2485
headers = []
for f in libfoo_SRC:
f_h = f[:-3] + ".h"
if os.path.isfile(f_h):
libfoo_SRC.append(f_h)
Import("build_env")
libh1ksim = build_env.StaticLibrary(
target = "foo",
source = libfoo_SRC)
build_env.Default(libfoo)
Now changes to headers that cause source files to rebuild will cause libfoo.a to get rebuilt and any target that link with libfoo.a.
weegreenblobbie said at 2013-02-26 09:50:04
Oops, previous solution breaks on windows.
Need to add header explicitly to depends list:
import os
libfoo_SRC = """
file1.cc
file2.cc
...
""".split()
# Hack / workaround scons bug 2485:
http://scons.tigris.org/issues/show_bug.cgi?id=2485
libfoo_HEADERS = []
for f in libfoo_SRC:
f_h = f[:-3] + ".h"
if os.path.isfile(f_h):
libfoo_HEADERS.append(f_h)
Import("build_env")
libfoo = build_env.StaticLibrary(
target = "foo",
source = libfoo_SRC)
build_env.Default(libfoo)
build_env.Depends(libfoo, libfoo_HEADERS)
dirkbaechle said at 2014-05-18 02:49:43
reassigning issue
Votes for this issue: 7.
jcoffland attached SCons-Static_Library_Failed_Change_Detection.tar.bz2 at 2009-09-21 17:44:13.
Simple example of static library change detection bug
Reran this experiment after the conftest change from #4161 and it seems to work - as long as the change you make to a library source is a real change, that causes the content sum of the object file to change. My first try was to change a comment - the object file rebuilt, but had the same md5sum, so nothing to relink against.
So perhaps this is fixed?
