Bear spawns an endless number of processes on FreeBSD

[Jehops]

After this commit on FreeBSD's main branch, bear will indefinitely spawn new processes.

root@15amd64-default:~ # uname -a
FreeBSD 15amd64-default 15.0-CURRENT FreeBSD 15.0-CURRENT 1500008 amd64

root@15amd64-default:~ # pkg info bear | head -11
bear-3.1.3_4
Name           : bear
Version        : 3.1.3_4
Installed on   : Tue Jan  2 13:54:27 2024 UTC
Origin         : devel/bear
Architecture   : FreeBSD:15:amd64
Prefix         : /usr/local
Categories     : devel
Licenses       : GPLv3+
Maintainer     : [email protected]
WWW            : https://github.com/rizsotto/Bear

root@15amd64-default:~ # bear --verbose -- ls bear_ls_output_freebsd15.txt Until SIGKILL is sent, new process will be spawned indefinitely.

Bear runs as expected on FreeBSD systems built prior to that commit.

root@14amd64-default:~ # uname -a
FreeBSD 14amd64-default 14.0-RC4 FreeBSD 14.0-RC4 amd64

root@14amd64-default:~ # pkg info bear | head -11
bear-3.1.3_4
Name           : bear
Version        : 3.1.3_4
Installed on   : Tue Jan  2 13:38:57 2024 UTC
Origin         : devel/bear
Architecture   : FreeBSD:14:amd64
Prefix         : /usr/local
Categories     : devel
Licenses       : GPLv3+
Maintainer     : [email protected]
WWW            : https://github.com/rizsotto/Bear

root@14amd64-default:~ # bear --verbose -- ls bear_ls_output_freebsd14.txt

[kittybwained]

I just upgraded to 14.0-STABLE and I'm getting the same issue, just endless wrappers spawning. If I run with --force-wrapper it seems to run the Makefile and exit, but it doesn't write anything in compile_commands.json.

[rizsotto]

Thanks guys for the reporting... Yes, if the newly introduced execve is causing the issue. (The fact that the posix_spawn is calling execve is the problem for this application.)

The potential workarounds are not easy with my time budget this time... Static linking of the wrapper could fix that. (A lot of work on the build system, which will not be as portable as it is today.) Instead of calling posix_spawn the wrapper should call the execve directly. (Seems like the cheapest solution.)

Using the --force-wapper is the no-code-change workaround. This mode of the interception is more depending on the build system... This page might be helpful to set it up.

[Jehops]

Unfortunately, --force-wrapper doesn't seem to be a viable workaround. Using bear --force-wrapper -- make on a project gives a compile_commands.json that only contains [].

[rizsotto]

I think it depends on the project @Jehops .

Three things have to match: the makefile should call the compiler not with an absolute path, and the compiler should have a matching wrapper (you can create a soft link), and if the compiler name is not recognized you need to register in the config file.

The link I've posted helps this to set up.

If you have a specific problem I can try to help you, but need more details.

[memreflect]

I just upgraded to 14.0-STABLE and I'm getting the same issue, just endless wrappers spawning. If I run with --force-wrapper it seems to run the Makefile and exit, but it doesn't write anything in compile_commands.json.

I'm running 14.1-RELEASE, and it turns out that the FreeBSD package maintainer does not install any of the wrapper symlinks, preventing --force-wrapper from working as intended.

To make --force-wrapper work, you can either create the necessary symlinks to ../wrapper inside /usr/local/lib/bear/wrapper.d yourself, or you can create a directory of your choice and pass that directory as the argument to Bear's --wrapper-dir flag:

$ rm -f foo foo.c compile_commands.json
$ echo 'int main(void){return 0;}' > foo.c
$ mkdir wrappers
$ ln -sv /usr/local/lib/bear/wrapper wrappers/clang16
wrappers/clang16 -> /usr/local/lib/bear/wrapper
$ bear --force-wrapper --wrapper-dir wrappers -- make CC=clang16 foo
clang16 -O2 -pipe  foo.c  -o foo
$ cat compile_commands.json
[
  {
    "arguments": [
      "/usr/local/bin/clang16",
      "-c",
      "-O2",
      "-pipe",
      "-o",
      "foo",
      "foo.c"
    ],
    "directory": "/tmp/bear.zDjlx5UdA8",
    "file": "/tmp/bear.zDjlx5UdA8/foo.c",
    "output": "/tmp/bear.zDjlx5UdA8/foo"
  }
]

The --wrapper-dir flag is currently undocumented, but it's the only alternative to compiling Bear from source when someone doesn't have superuser access. Hopefully the wrapper workaround helps people until this issue can be resolved.

(P.S. I'm not sure why the -c flag was added to the argument list... Weird!)

[purpledan]

I have been following this bug report since 14.1-RELEASE (That is when bear stopped working for me). The fix proposed above by @memreflect works for me.

I had to create a symlink to ../wrapper inside /usr/local/lib/bear/wrapper.d. Then it works by simply calling bear with --force-wrapper. This creates a compile_commands.json exactly the same to one created on 14.0-RELEASE.

The non superuser access version did not work for me, I am not sure why.

I hope this helps. Maybe the port maintainer should do this by default?

[Jehops]

I can add something like this to the port to create a bunch of symlinks.

.for l in c++ cc clang++ clang++11 clang++12 clang++13 clang++14 clang++15 \
	clang++16 clang++17 clang++18 clang++19 clang++20 clang clang11 \
	clang12 clang13 clang14 clang15 clang16 clang17 clang18 clang19 \
	clang20 cpp gcc10 g++11 g++12 g++13 g++14 g++15 gcc11 gcc12 gcc13 \
	gcc14 gcc15
	${RLN} ${STAGEDIR}${PREFIX}/lib/bear/wrapper \
		${STAGEDIR}${PREFIX}/lib/bear/wrapper.d/${l}
.endfor

I can also add a note to let users know that --force-wrapper is a necessary workaround on FreeBSD 14.1+ due to this bug.

Do you suggest any other symlinks?

[derekschrock]

On Fri, Aug 16, 2024 at 01:26:34PM EDT, Joseph Mingrone wrote:

I can add something like this to the port to create a bunch of symlinks.

You might want to look at ccache port it does the same thing for /usr/local/libexec/ccache/

[Jehops]

Thanks. I extracted some bits from the ccache port to come up with:

CLANG_COMPILERS =	33 34 35 36 37 38 39 40 50 60 70 80 90 10 11 12 13 14 \
			15 16 17 18 19 20
GNU_COMPILERS=		34 42 43 44 45 46 47 48 49 5 6 7 8 9 10 11 12 13 14 15
COMPILERS=		cc c++ CC clang clang++ ${CLANG_COMPILERS:S|^|clang|} \
			${CLANG_COMPILERS:S|^|clang++|} \
			${CLANG_COMPILERS:S|^|cpp|} \
			gcc g++ ${GNU_COMPILERS:S|^|gcc|} \
			${GNU_COMPILERS:S|^|g++|} \
			${GNU_COMPILERS:S|^|cpp|} icc icpc llvm-gcc llvm-c++ \
			llvm-g++

...

post-install:
...
.for l in ${COMPILERS}
	${RLN} ${STAGEDIR}${PREFIX}/lib/bear/wrapper \
		${STAGEDIR}${PREFIX}/lib/bear/wrapper.d/${l}
.endfor

That's a fairly exhaustive list of C/C++ compilers. I notice that, e.g., the Arch bear package [0] creates links for other binaries. I'm open to thoughts about which are useful.

[0] See Package Contents at https://archlinux.org/packages/extra/x86_64/bear/.