WIP vpath build...

[elliefm]

[elliefm]

This seems to be working now, but if I build an image and then:

$git clean -xfd
$dar start --image=foo
$dar build
$dar test
$dar prune

..., I expect that due to the VPATH build, nothing should have changed in my source directory. And yet:

$git clean -xfd
Removing Makefile.in
Removing aclocal.m4
warning: failed to remove autom4te.cache/traces.0: Permission denied
warning: failed to remove autom4te.cache/output.0: Permission denied
warning: failed to remove autom4te.cache/output.3: Permission denied
warning: failed to remove autom4te.cache/requests: Permission denied
warning: failed to remove autom4te.cache/output.2: Permission denied
warning: failed to remove autom4te.cache/traces.1: Permission denied
warning: failed to remove autom4te.cache/traces.3: Permission denied
warning: failed to remove autom4te.cache/output.1: Permission denied
warning: failed to remove autom4te.cache/traces.2: Permission denied
Removing cassandane/cassandane.ini
Removing cassandane/utils/crash
Removing cassandane/utils/crash.o
Removing cassandane/utils/gdbtramp
Removing cassandane/utils/gdbtramp.o
Removing cassandane/utils/lemming
Removing cassandane/utils/lemming.o
Removing cassandane/utils/syslog.o
Removing cassandane/utils/syslog.so
Removing cassandane/utils/syslog_probe
Removing cassandane/utils/syslog_probe.o
Removing cmulocal/libtool.m4
Removing cmulocal/ltoptions.m4
Removing cmulocal/ltsugar.m4
Removing cmulocal/ltversion.m4
Removing cmulocal/lt~obsolete.m4
Removing compile
Removing config.guess
Removing config.h.in
Removing config.sub
Removing configure
Removing depcomp
Removing install-sh
Removing ltmain.sh
Removing missing
Removing ylwrap

There's a handful of files written to the srcdir still. The cassandane files don't surprise me, but the rest are confusing because make distcheck works, and make distcheck supposedly proves that you can build from a read only source directory (among other things). I guess I haven't updated cyd test to know about the separate builddir yet, maybe that's the culprit.

[elliefm]

Oh right, most (all?) of those were generated by autoreconf. distcheck isn't bothered by them, because from distcheck's point of view these were already in the source directory (and expected to be) when it started.

[rjbs]

This all seems very plausible to me. "No objection." But definitely Matthew may have more useful feedback. If not, great, ship it! (I did not test with an image using this, and trust you did!)

[elliefm]

(I did not test with an image using this, and trust you did!)

Yeah, a lot! One of the things that stalled this a few months ago was I needed to fix a bunch of cunit incompatibilities that were preventing it from working. I did that, but then forgot that that was what was blocking this, and kept ignoring this one as "blocked by something..."

[wolfsage]

One downside to this is you can no longer edit a file, then do make -j 8 install. Instead you need make -C /usr/cyrus/build -j 8 install. I think this is fine, really cyd rebuild or something should be a command.

I suspect @brong might have thoughts here depending on his workflow.

[wolfsage]

I think my biggest concern is this means a non-vpath build is no longer exercised - isn't that important?

I think I'd prefer --vpath as an option to cyd build etc, or, if we think vpath is the most common case, --no-vpath as an option, and we have a ci build that builds that....

[elliefm]

@wolfsage,

One downside to this is you can no longer edit a file, then do make -j 8 install. Instead you need make -C /usr/cyrus/build -j 8 install. I think this is fine, really cyd rebuild or something should be a command.

Hmm, that's a good point. I mostly think of cyd as "the thing that CI uses", not so much as a daily workflow tool. But it's meant to be that too, and this makes that usage less ergonomic (though yeah a cyd rebuild would solve that issue).

I think my biggest concern is this means a non-vpath build is no longer exercised - isn't that important?

Probably not. A non-vpath build is really just a vpath build using the same directory for srcdir and builddir, which is the usual assumption. Since vpath builds are the more complex case, we want to exercise them to make sure our usual assumptions haven't broken them. But I think as long as a vpath build succeeds, so would a non-vpath build.

There are probably things you could do such that a vpath build succeeds but non-vpath fails, but I think you'd have to be trying to do that, it's not something you'd do by accident. The other way round is a very common mistake.

I think I'd prefer --vpath as an option to cyd build etc, or, if we think vpath is the most common case, --no-vpath as an option, and we have a ci build that builds that....

Hmm, perhaps that would complicate things more than necessary...

My goals here were approximately:

1. stop "dar foo" from polluting my source tree with root-owned build files
2. have CI regularly running VPATH builds, as a substitute for make distcheck

The former hasn't been completely achieved, due to the autoreconf outputs. The latter would be unnecessary if we had CI running make distcheck, and I already have a task open for that. So maybe I'll just close this one without merging. What do you think?

[elliefm]

Actually, I suppose most of this infrastructure could be kept for either style of build. The only thing that changes between vpath/non-vpath is whether $build_dir is initialised to '/usr/cyrus/build' or $root.

So maybe instead of --vpath or --no-vpath options, it could be a --builddir=path option which defaults to $root but can be overridden.