Make failed when building out-of-source

[gyatpear]

Describe the bug

Make failed building out-of-source, because of the disabling rebuilding src/lexer.c and src/parser.c.

To Reproduce

Do this:

git submodule update --init
autoreconf -i
mkdir build
cd build
../configure --with-oniguruma=builtin
make

And get the error message:

  YACC     src/lexer.c
NOT building lexer.c!
  CC       src/lexer.lo
cc: error: no such file or directory: 'src/lexer.c'
cc: error: no input file

This is because the implicit rules of GNU Make. The existence of $(srcdir)/src/lexer.l will disable the VPATH $(srcdir)/src/lexer.c to be the input file for src/lexer.o. So as src/parser.c.

However, if configure without maintainer-mode enabled, Makefile.am has a rule to disable %.l: %.c and %.y: %.c generation.

And if you are not building out-of-source, src/lexer.c and src/parser.c should be fine as input, instead of VPATH $(srcdir)/src/lexer.c and $(srcdir)/src/parser.c.

Expected behavior

jq to be built.

Environment (please complete the following information):

• OS and Version: Ubuntu 24.04.1 LTS
• jq version: jq-1.8.1 (git tag)
• GNU Make: 4.3

[sudhackar]

I'm not able to repro this. With this dockerfile

FROM ubuntu:24.04

# Install essential build tools and dependencies
RUN apt-get update && apt-get install -y \
    git \
    make \
    gcc \
    autoconf \
    automake \
    libtool \
    flex \
    bison \
    pkg-config \
    build-essential \
    && rm -rf /var/lib/apt/lists/*

# Clone jq repository and checkout the jq-1.8.1 tag
RUN git clone https://github.com/jqlang/jq.git /jq
WORKDIR /jq
RUN git checkout jq-1.8.1
RUN git submodule update --init

RUN autoreconf -i

# Already exists in jq
# RUN mkdir /jq/build
WORKDIR /jq/build
RUN ../configure --with-oniguruma=builtin

# This should trigger the error:
RUN make || true

CMD ["/bin/bash"]

Now when you run

$ docker build -t jq-oos-bug .
$ docker run --rm --name jq-oos-bug -it jq-oos-bug /jq/build/jq --version
jq-1.8.1
$ docker run --rm --name jq-oos-bug -it jq-oos-bug make --version
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

This builds the image with jq-1.8.1 and we can see build is correct

[gyatpear]

After re-built the docker, as well as doing the same clone-configure-make on my host machine, many many times, I found the problem.

It is because the date of $(srcdir)/src/lexer.c is (sometimes, due to git) older than $(srcdir)/src/lexer.l. And the error can be resolved by touch src/lexer.c.

I have just done ls -al the src directory after git clone, saw no differences. Then I realized I should use stat(1) to show the precise timestamp and they have milliseconds differences. git just put the files to the correct path, and does not matter the timestamp.

[itchyny]

Would you suggest a way to resolve the issue?

[gyatpear]

Would you suggest a way to resolve the issue?

Summary:

1. Manually touch the generated file after git-clone or something. The tarball generated by make dist may not have this issue since it saved the timestamp.
2. Add support for CMake or Meson (the jq project is so simple that I think it is not hard).

After digging in the Autotools, my conclusion is that Autotools is not fully designed for out-of-source build. (or maybe I am not familiar enough to Autotools, but I have tried almost everything mentioned by the doc).

I have found Automake provides no way to remove .l and .y from .SUFFIXES. This causes make to always do the .l.c and .y.c rule.

https://github.com/jqlang/jq/blob/4357b6154e35a9984152f3eef2759c9b118db719/configure.ac#L24-L25

At first, I think AM_PROG_LEX will enable the missing script and I try to resolve the "header issue" with AM_PROG_LEX. Then I found no help for this issue. The missing script does not keep make from doing the .l.c and .y.c rule.

Now I have no idea for editing the current code since it works most of the time. My suggestions are the above.