oss-fuzz icon indicating copy to clipboard operation
oss-fuzz copied to clipboard

[CFL] Set CLUSTERFUZZLITE variable when building and running

Open jonathanmetzman opened this issue 3 years ago • 7 comments

Related: https://github.com/google/clusterfuzzlite/issues/77

jonathanmetzman avatar Jan 26 '22 16:01 jonathanmetzman

CC @evverx

jonathanmetzman avatar Jan 26 '22 16:01 jonathanmetzman

@jonathanmetzman thanks! I think the commit message should point to https://github.com/google/clusterfuzzlite/issues/77 instead of https://github.com/google/oss-fuzz/pull/77.

FWIW Not that it matters but I think it would be better if CIFUZZ wasn't set when it's run on CFLite to make it possible for build scripts to really tell CIFuzz and CFLite apart. For example, CIFuzz automatically downloads public OSS-Fuzz corpora so build scripts don't have to do that but when they are run on CFLite it would probably make sense to download them manually. Then again, it covers that "weird" usecase where both CIFuzz and CFLite are used together to cover "main" branches and forks.

evverx avatar Jan 26 '22 16:01 evverx

@jonathanmetzman thanks! I think the commit message should point to google/clusterfuzzlite#77 instead of #77.

FWIW Not that it matters but I think it would be better if CIFUZZ wasn't set when it's run on CFLite to make it possible for build scripts to really tell CIFuzz and CFLite apart. For example, CIFuzz automatically downloads public OSS-Fuzz corpora so build scripts don't have to do that but when they are run on CFLite it would probably make sense to download them manually. Then again, it covers that "weird" usecase where both CIFuzz and CFLite are used together to cover "main" branches and forks.

I was hoping this wouldn't be an issue. Lemme try and change this.

jonathanmetzman avatar Jan 26 '22 16:01 jonathanmetzman

systemd has started using both CIFuzz and CFLite so issues like that are no longer theoretical :-)

evverx avatar Jan 29 '22 18:01 evverx

I tried to get it around in https://github.com/systemd/systemd/pull/22302 but apparently I can't easily pass environment variables to my build script because it doesn't seems to be possible to pass them on to docker with -e

evverx avatar Jan 29 '22 22:01 evverx

I passed that environment variable using .clusterfuzzlite/Dockerfile

diff --git a/.clusterfuzzlite/Dockerfile b/.clusterfuzzlite/Dockerfile
index bdb0378ebc..a69d8bfb70 100644
--- a/.clusterfuzzlite/Dockerfile
+++ b/.clusterfuzzlite/Dockerfile
@@ -1,4 +1,5 @@
 FROM gcr.io/oss-fuzz-base/base-builder@sha256:14b332de0e18683f37386eaedbf735bc6e8d81f9c0e1138d620f2178e20cd30a
+ENV MERGE_WITH_OSS_FUZZ_CORPORA yes
 COPY . $SRC/systemd
 WORKDIR $SRC/systemd
 COPY tools/oss-fuzz.sh $SRC/build.sh

evverx avatar Jan 29 '22 22:01 evverx

I think CFLite should pass UPLOAD_BUILD to build_fuzzers to make it possible to exclude those large corpora from artifacts. Without that environment variable I can't skip that step with something like

if [[ "$MERGE_WITH_OSS_FUZZ_CORPORA" == "yes" ]]; then

    # When the latest builds are uploaded by CFLite the large OSS-Fuzz corpora
    # should be excluded regardless of whether MERGE_WITH_OSS_FUZZ_CORPORA
    # is set to "yes" or not.
    [[ "$UPLOAD_BUILD" == "true" ]] && exit 0
    ....

evverx avatar Jan 30 '22 02:01 evverx