differenet database generated while using parallel build(-jN)

[kaijajan]

I noticed different compilation database generated while using parallel build. I use the following commands to generate build_log.txt

make -w | tee build_log.txt
compiledb < build_log.txt

make -w -j8 | tee build_log.txt
compiledb < build_log.txt

I fetch part of the built log did not generated in compile_commands.json and use both of them run with compiledb. compiledb generates nothing between squares if build log is parallel one. but done well with normal one.

part of built log without -jN, build_log_j1.txt

make[2]: Entering directory '/home/user/workspaces/bsp/projects/ats2/app/cli'
cp -f -R man /home/user/workspaces/bsp/projects/ats2/target/mnt/
make[3]: Entering directory '/home/user/workspaces/bsp/projects/ats2/app/cli/src'
armv5-linux-gcc -c -I/home/user/workspaces/bsp/projects/ats2/app/include  -Os -Wall cli.c
armv5-linux-gcc -c -I/home/user/workspaces/bsp/projects/ats2/app/include  -Os -Wall cmd.c
armv5-linux-gcc -o cli cli.o cmd.o -Os -Wall    -L/home/user/workspaces/bsp/projects/ats2/target/usr/lib -lmisc -lperiphery -lcommon -lobj -lreadline
armv5-linux-strip cli
armv5-linux-gcc -c -I/home/user/workspaces/bsp/projects/ats2/app/include  -Os -Wall cliReadFile.c
armv5-linux-gcc -o cliReadFile cliReadFile.o cmd.o -Os -Wall    -L/home/user/workspaces/bsp/projects/ats2/target/usr/lib -lmisc -lperiphery -lcommon -lobj -lreadline
armv5-linux-strip cliReadFile
make[3]: Leaving directory '/home/user/workspaces/bsp/projects/ats2/app/cli/src'
make[2]: Leaving directory '/home/user/workspaces/bsp/projects/ats2/app/cli'
make[2]: Entering directory '/home/user/workspaces/bsp/projects/ats2/app/cli'
make[3]: Entering directory '/home/user/workspaces/bsp/projects/ats2/app/cli/src'
for i in cli cliReadFile ; do \
    cp -f $i /home/user/workspaces/bsp/projects/ats2/target/bin || exit 1; \
done
make[3]: 'install' is up to date.
make[3]: Leaving directory '/home/user/workspaces/bsp/projects/ats2/app/cli/src'
make[2]: Leaving directory '/home/user/workspaces/bsp/projects/ats2/app/cli'

part of built log with -jN, build_log_j8.txt

make[2]: Entering directory '/home/archer/workspaces/bsp/projects/ats2/app/cli'
cp -f -R man /home/archer/workspaces/bsp/projects/ats2/target/mnt/
make[3]: Entering directory '/home/archer/workspaces/bsp/projects/ats2/app/cli'
armv5-linux-gcc -c -I/home/archer/workspaces/bsp/projects/ats2/app/include  -Os -Wall cli.c
armv5-linux-gcc -c -I/home/archer/workspaces/bsp/projects/ats2/app/include  -Os -Wall cmd.c
armv5-linux-gcc -c -I/home/archer/workspaces/bsp/projects/ats2/app/include  -Os -Wall cliReadFile.c
armv5-linux-gcc -o cli cli.o cmd.o -Os -Wall    -L/home/archer/workspaces/bsp/projects/ats2/target/usr/lib -lmisc -lperiphery -lcommon -lobj -lreadline
armv5-linux-gcc -o cliReadFile cliReadFile.o cmd.o -Os -Wall    -L/home/archer/workspaces/bsp/projects/ats2/target/usr/lib -lmisc -lperiphery -lcommon -lobj -lreadline
armv5-linux-strip cli
armv5-linux-strip cliReadFile
make[3]: Leaving directory '/home/archer/workspaces/bsp/projects/ats2/app/cli/src'
make[2]: Leaving directory '/home/archer/workspaces/bsp/projects/ats2/app/cli'
make[2]: Entering directory '/home/archer/workspaces/bsp/projects/ats2/app/cli'
make[3]: Entering directory '/home/archer/workspaces/bsp/projects/ats2/app/cli'
for i in cli cliReadFile ; do \
    cp -f $i /home/archer/workspaces/bsp/projects/ats2/target/bin || exit 1; \
done
make[3]: 'install' is up to date.
make[3]: Leaving directory '/home/archer/workspaces/bsp/projects/ats2/app/cli/src'
make[2]: Leaving directory '/home/archer/workspaces/bsp/projects/ats2/app/cli'

here are the commands i use to generate database

compiledb -p build_log_j1.txt -o j1.json
compiledb -p build_log_j8.txt -o j8.json

two files are basically the same except the order

[RPCMoritz]

I think this is due to parallel builds generating a disordered output. See how below all -c/-o/strip are clustered, whereas above they are in the correct order for each artifact. If you use -Bwnk flags for make, you should be able to generate the output with -j1 in a reasonable amount of time - therefore I would recommend that you avoid unnecessary concurrency. compiledb is quite fragile when it comes to parsing those make outputs.

[lyonlh]

@kaijajan The make's option --output-sync would help you.