chronos: Add a ptrace-based tracer.

[oliverchang]

This is based on our past SystemSan work.

This will recursively find the command that builds the fuzz target, and print out the environment variables and commands for building the target.

Usage:

$ tracer <fuzz_target_source_name> <output_path> <build command>

Output (written to <output_path>):

export ENV=val
export ENV=val

cd /path/to/cwd
clang -o fuzz_target ...

(Known) remaining work:

• Stop the entire build process when the fuzz target build command is found. Some projects delete entire source directories after the build is done: https://github.com/google/oss-fuzz/blob/c8bca3bd4a57e6128f0780725aca476e7e6cbd33/projects/openssl/build.sh#L75
• Properly shell escape the environment variables we extract

[oliverchang]

Note: this is not ready to land. This is missing some functionality to figure out the list of all commands to run past the command which builds the fuzz target source. This is going to be hard to do, as it's hard to distinguish which commands to record in a way that ensures the build steps are the same (because commands can themselves call execve and execute more-subcommands) and we can't just blindly re-run everything.

The alternative is addressing the known issue #1 in the PR description: namely stopping and checkpointing the container at this exact point (https://docs.docker.com/reference/cli/docker/checkpoint/). But it's not clear how reliable CRIU is for this use case.

[jonathanmetzman]

Note: this is not ready to land. This is missing some functionality to figure out the list of all commands to run past the command which builds the fuzz target source. This is going to be hard to do, as it's hard to distinguish which commands to record in a way that ensures the build steps are the same (because commands can themselves call execve and execute more-subcommands) and we can't just blindly re-run everything.

The alternative is addressing the known issue #1 in the PR description: namely stopping and checkpointing the container at this exact point (https://docs.docker.com/reference/cli/docker/checkpoint/). But it's not clear how reliable CRIU is for this use case.

There's no way we can see what reads the source file and/or writes the binary (will fail if linking done at seperate stage) and pause then? I think really the goal is to pause at the source code reading so this would seem promising.

[oliverchang]

Note: this is not ready to land. This is missing some functionality to figure out the list of all commands to run past the command which builds the fuzz target source. This is going to be hard to do, as it's hard to distinguish which commands to record in a way that ensures the build steps are the same (because commands can themselves call execve and execute more-subcommands) and we can't just blindly re-run everything. The alternative is addressing the known issue #1 in the PR description: namely stopping and checkpointing the container at this exact point (https://docs.docker.com/reference/cli/docker/checkpoint/). But it's not clear how reliable CRIU is for this use case.

There's no way we can see what reads the source file and/or writes the binary (will fail if linking done at seperate stage) and pause then? I think really the goal is to pause at the source code reading so this would seem promising.

Yeah, good point. We could just try to replay those 2 commands: compiling the source and linking the binary. My main concern there was if it's common to have intermediate steps in between the compilation and linking have to be replayed also.

[oliverchang]

Thinking more about fast rebuilds that work for all OSS-Fuzz, I'm thinking about another approach, where we build the entire project, and to rebuild, we just re-execute the entire build script, but we stub out cmake, ./configure etc, ./autogen.sh. Then we just rely on the build system to figure out what to rebuild when the relevant make command is executed. Another benefit here is we can re-use this for fast rebuilding of OSS-Fuzz projects for other use cases (e.g. validating patches).

There are likely still edge cases here:

• e.g. openssl that deletes their entire /src directory after building: https://github.com/google/oss-fuzz/blob/c8bca3bd4a57e6128f0780725aca476e7e6cbd33/projects/openssl/build.sh#L75. But we can just remove this on a case by case basis (and increase the underlying disk for such projects).

[oliverchang]

Closing this in favour of https://github.com/google/oss-fuzz/pull/12608.

There may still be benefit in exploring a ptrace/ebpf approach to rebuilding projects that simply reruns the build, but blocks/modifies certain commands at runtime (rather than trying to statically parse the set of commands we should re-run). This depends on how well https://github.com/google/oss-fuzz/pull/12608 works across all C/C++ projects we have today.

@DavidKorczynski