Makefiles are missing targets, can't run `make distclean`

[gsteemso]

Description When building Perl, the generated Makefile is missing targets for (at a minimum) config.sh and makedepend_file. As these are referenced from assorted secondary targets that are processed during processing of the clean or distclean targets, those targets cause make to fail with an error 2.

Steps to Reproduce At any point after having performed the ./Configure step, attempt to make distclean. It will fail.

Expected behavior The assorted files generated by a ./Configure run should be deleted, resetting the build directory to a state suitable for a differently-configured build.

Perl configuration

Site configuration information for perl 5.38.2:

Configured by gsteemso at Sun May 26 18:21:43 PDT 2024.

Summary of my perl5 (revision 5 version 38 subversion 2) configuration:
   
  Platform:
    osname=darwin
    osvers=9.8.0
    archname=darwin-thread-multi-2level
    uname='darwin nosferalto.local 9.8.0 darwin kernel version 9.8.0: wed jul 15 16:57:01 pdt 2009; root:xnu-1228.15.4~1release_ppc power macintosh powerpc '
    config_args='-des -Dprefix=/Users/Shared/Brewery/Cellar/perl/5.38.2 -Uvendorprefix= -Dprivlib=/Users/Shared/Brewery/Cellar/perl/5.38.2/lib -Darchlib=/Users/Shared/Brewery/Cellar/perl/5.38.2/lib -Dman1dir=/Users/Shared/Brewery/Cellar/perl/5.38.2/share/man/man1 -Dman3dir=/Users/Shared/Brewery/Cellar/perl/5.38.2/share/man/man3 -Dman3ext=3pl -Dsitelib=/Users/Shared/Brewery/Cellar/perl/5.38.2/lib/site_perl -Dsitearch=/Users/Shared/Brewery/Cellar/perl/5.38.2/lib/site_perl -Dsiteman1dir=/Users/Shared/Brewery/Cellar/perl/5.38.2/share/man/man1 -Dsiteman3dir=/Users/Shared/Brewery/Cellar/perl/5.38.2/share/man/man3 -Dperladmin=none -Dstartperl=#!/usr/local/opt/perl/bin/perl -Duseshrplib -Duselargefiles -Dusenm -Dusethreads -Acppflags=-m64 -Duse64bitall'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cc'
    ccflags ='-std=gnu99 -fno-common -DPERL_DARWIN -mmacosx-version-min=10.5 -DNO_POSIX_2008_LOCALE -arch ppc64 -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_FORTIFY_SOURCE=2'
    optimize='-O3'
    cppflags='-arch ppc64 -m64 -std=gnu99 -fno-common -DPERL_DARWIN -mmacosx-version-min=10.5 -DNO_POSIX_2008_LOCALE -arch ppc64 -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 (Apple Inc. build 5577)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=87654321
    doublekind=4
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=6
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc -arch ppc64'
    ldflags =' -mmacosx-version-min=10.5 -arch ppc64  -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib
    libs=-lpthread -ldbm -ldl -lm -lutil -lc
    perllibs=-lpthread -ldl -lm -lutil -lc
    libc=/usr/lib/libc.dylib
    so=dylib
    useshrplib=true
    libperl=libperl.dylib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=bundle
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' -mmacosx-version-min=10.5 -bundle -undefined dynamic_lookup  -L/usr/local/lib -fstack-protector'


---
@INC for perl 5.38.2:
    /Users/Shared/Brewery/Cellar/perl/5.38.2/lib/site_perl
    /Users/Shared/Brewery/Cellar/perl/5.38.2/lib

---
Environment for perl 5.38.2:
    DYLD_LIBRARY_PATH (unset)
    HOME=/private/tmp/perl20240526-62600-1ky2zf9/perl-5.38.2/.brew_home
    LANG=en_CA.UTF-8
    LANGUAGE (unset)
    LC_ALL=en_US.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/Users/Shared/Brewery/Library/ENV/4.3:/usr/local/opt/xz/bin:/usr/bin:/bin:/usr/sbin:/sbin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[mauke]

At any point after having performed the ./Configure step, attempt to make distclean. It will fail.

Did that. It didn't fail (in either the 5.38.2 release or blead).

[gsteemso]

I apologize that I could not submit a patch for this. I am unable to make sense of the .SH file that produces the Makefile.

[gsteemso]

At any point after having performed the ./Configure step, attempt to make distclean. It will fail.

Did that. It didn't fail (in either the 5.38.2 release or blead).

Iiiinteresting. It therefore has to be something in the .SH file which is being correctly triggered on your system but not on mine.

[gsteemso]

If I edit the GNUmakefile directly (to tell it how to generate makedepend_file, and to insert a fake entry that does nothing for the config.sh target), it whines repeatedly that it can't find config.sh but otherwise succeeds.

[jkeenan]

If I edit the GNUmakefile directly (to tell it how to generate makedepend_file, and to insert a fake entry that does nothing for the config.sh target), it whines repeatedly that it can't find config.sh but otherwise succeeds.

1. Like @mauke, I configured, then called make distclean, and had no problem getting my directory back to a clean state.
2. AFAICT, both config.sh and makedepend_file are generated by ./Configure. So calling either as a make target once ./Configure has run is not necessary and (on my system, at least), simply results in a "Nothing to be done for __" message.
3. Your system info indicates you're on Darwin. Why, then, are you playing around with GNUmakefile which is intended for Win32?

[Leont]

Your system info indicates you're on Darwin. Why, then, are you playing around with GNUmakefile which is intended for Win32?

Mac uses GNUmakefile too (because the file system is case insensitive like on Windows).

[gsteemso]

If I edit the GNUmakefile directly (to tell it how to generate makedepend_file, and to insert a fake entry that does nothing for the config.sh target), it whines repeatedly that it can't find config.sh but otherwise succeeds.

1. Like @mauke, I configured, then called make distclean, and had no problem getting my directory back to a clean state.
2. AFAICT, both config.sh and makedepend_file are generated by ./Configure. So calling either as a make target once ./Configure has run is not necessary and (on my system, at least), simply results in a "Nothing to be done for __" message.
3. Your system info indicates you're on Darwin. Why, then, are you playing around with GNUmakefile which is intended for Win32?

1. As above, that means Makefile is generated correctly on your machine and not on mine.
2. Neither SHOULD be necessary, no, but when I run make distclean it crashes because there is no target for config.sh, and when I run make clean it crashes because there is no target for distclean_file. I can't call either one directly, because Makefile is generated wrong and THERE IS NO TARGET FOR EITHER ON MY MACHINE.
3. As stated, on Mac OS X it can't generate makefile because Makefile would be overwritten, so it makes GNUmakefile. That's what GNUmakefile is for.

[gsteemso]

Does anyone know how to make sense of Makefile.SH? I can't debug it because I can't figure out how it works. It appears to be what generates Makefile after Configure has generated config.sh, but all of its contents appear to spit out lumps of shell script, not make script!

[mauke]

echo "Extracting $Makefile (with variable substitutions)"
$spitshell >$Makefile <<!GROK!THIS!
...

[gsteemso]

Further observations:

1. make config.sh et sim are supposed to return "nothing to be done for ..." when asked to Make an up-to-date file, so that's not a useful data point.

2. the problem evaporates if I make -j1 [dist]clean instead of letting it choose a default number of parallel jobs. I still can't figure out what it's whining about in the problem case, unless perhaps every child process Make spawns inherently re-parses [GNU]makefile as a first step? Since the makefile depends on other things, it will believe itself Not Up To Date if those other things (such as config.sh) are absent. I could see that being a thing that happens if each child process acts as though invoked from scratch with a command line of make _sub-target_.

[mauke]

Confirmed:

$ make -j6 distclean 
rm -f *.o *.a all perlmain.c opmini.c perlmini.c universalmini.c generate_uudmap uudmap.h bitcount.h mg_data.h
rm -rf host
rmdir .depending
rm -f t/test_state
rmdir: failed to remove '.depending': No such file or directory
make: [makefile:643: _mopup] Error 1 (ignored)
rm *.depends makedepend_file
rm -f config.sh cppstdin Policy.sh extras.lst
rm -f MANIFEST.srt
rm: cannot remove '*.depends': No such file or directory
make: [makefile:644: _mopup] Error 1 (ignored)
make: [makefile:645: _mopup] Error 1 (ignored)
make: [makefile:646: _mopup] Error 1 (ignored)
rm -f perl.exp ext.libs pod/perltoc.pod extra.pods pod/perl5382delta.pod pod/perlapi.pod pod/perlintern.pod pod/perlmodlib.pod pod/perluniprops.pod uni.data opmini.o perlmini.o universalmini.o pod/roffitall
rm -f perl.export perl.dll perl.libexp perl.map perl.def
rm -f *perl.xok
rm -f cygwin.c libperl*.def libperl*.dll cygperl*.dll *.exe.stackdump
rm -f perl miniperl libperl.a libperl.* microperl
rm -f config.arch config.over 
make[1]: Entering directory '/home/mauke/tmp/perl-5.38.2'
make[1]: *** No rule to make target 'config.sh', needed by 'Makefile'.  Stop.
make[1]: Leaving directory '/home/mauke/tmp/perl-5.38.2'
make: *** [makefile:743: _realcleaner] Error 2

[gsteemso]

I see that modern versions of Make allow specific compilation targets to be defined as .NOTPARALLEL, but I don't know when that started being possible, so I'm not sure it would be a useful thing to add to the Perl distribution. Is there a portable way to alert the user that parallel execution of target X will not work?

[gsteemso]

I can confirm that .NOTPARALLEL both exists and works on GNU make 3.81, though it's underdocumented; however, that far back, it is always absolute rather than optionally applying only to its prerequisites, which for that old version are ignored. Since most platforms presumably have or can get a more recent Make, I propose adding the line

.NOTPARALLEL: realclean clobber veryclean

to the generated makefile.

[Leont]

Or maybe we should clean up our cleanup logic. It' currently overly complicated for reasons that aren't clear to me at all (and that aren't documented).

It's currently spread over several targets, and those targets have no declared dependencies on one another. Clearly that is wrong. I have a feeling making this less smart is the better solution, really.

[gsteemso]

Good point. It appears that the different targets merely add stages to the same list of prerequisites, except for the part that gets transmitted to recursive-subdirectory makefiles.

[gsteemso]

This has been fixed in 5.40.0. Marking this closed.