add --stdin flag to use `find` for excluding of files, folders and mount points

[beckerr-rzht]

Fixes #42

[beckerr-rzht]

This is a merged version of #43

[beckerr-rzht]

Current version build from this RP: https://github.com/beckerr-rzht/log4j-detector/raw/release/log4j-detector-2021.12.29.jar

[pthoelken]

As always, thanks for your work in this repo @beckerr-rzht ! :)

Can you told me, which is here the best solution to call the script? Currently I'm using this like this: https://github.com/pthoelken/log4j-searcher/blob/55f263aad20ed29f36f3476cfebb1ee3c0524f3b/runner.sh#L58

but I'm not sure if it's work correctly. The script is done really fast. Can you give me your preffered method how do you would start the jar which should check the whole linux/mac beginning from /

I'll ask you because when I check with samples from https://github.com/mergebase/log4j-samples/tree/master/true-hits he told me this:

-- Skipping C:\home\username\log4j-samples\false-hits\log4j-api-2.14.1.jar - Not a regular file.
-- Skipping C:\home\username\log4j-samples\false-hits\log4j-core-2.12.2.jar - Not a regular file.
-- Skipping C:\home\username\log4j-samples\false-hits\log4j-core-2.12.3.jar - Not a regular file.
-- Skipping C:\home\username\log4j-samples\false-hits\log4j-core-2.12.4.jar - Not a regular file.
-- Skipping C:\home\username\log4j-samples\false-hits\log4j-core-2.15.0.jar - Not a regular file.
-- Skipping C:\home\username\log4j-samples\false-hits\log4j-core-2.16.0.jar - Not a regular file.
-- Skipping C:\home\username\log4j-samples\false-hits\log4j-core-2.17.0.jar - Not a regular file.
-- Skipping C:\home\username\log4j-samples\false-hits\log4j-core-2.17.1.jar - Not a regular file.
-- Skipping C:\home\username\log4j-samples\false-hits\log4j-core-2.3.1.jar - Not a regular file.
-- Skipping C:\home\username\log4j-samples\false-hits\log4j-core-2.3.2.jar - Not a regular file.

usually it should be like this: (show here for example)

false-hits/log4j-core-2.12.2.jar contains Log4J-2.x   == 2.12.2 _OKAY_
false-hits/log4j-core-2.12.3.jar contains Log4J-2.x   == 2.12.3 _OKAY_
false-hits/log4j-core-2.12.4.jar contains Log4J-2.x   == 2.12.4 _SAFE_
false-hits/log4j-core-2.15.0.jar contains Log4J-2.x   == 2.15.0 _OKAY_
false-hits/log4j-core-2.16.0.jar contains Log4J-2.x   == 2.16.0 _OKAY_
false-hits/log4j-core-2.17.0.jar contains Log4J-2.x   == 2.17.0 _OKAY_

Thanks a lot and happy new year!

[beckerr-rzht]

I use a script that looks more or less like this:

#!/bin/bash -e

tmpdir=$(mktemp -d)
cd "$tmpdir"

cleaner() {
    echo "* Removing $tmpdir"
    rm -rf "${tmpdir:-does-not-exist}"
}

trap cleaner INT TERM EXIT

detector="https://github.com/beckerr-rzht/log4j-detector/raw/release/log4j-detector-2021.12.29.jar"

m="$(dpkg --print-architecture 2>/dev/null || uname -m)"
case "$m" in
armsf) jre="https://cdn.azul.com/zulu-embedded/bin/zulu11.52.13-ca-jdk11.0.13-linux_aarch32sf.tar.gz" ;; # RPI
armhf) jre="https://cdn.azul.com/zulu-embedded/bin/zulu11.52.13-ca-jdk11.0.13-linux_aarch32hf.tar.gz" ;; # RPI
*64)   jre="https://cdn.azul.com/zulu/bin/zulu11.52.13-ca-jre11.0.13-linux_x64.tar.gz" ;; # 64 Bit
i?86)  jre="https://cdn.azul.com/zulu/bin/zulu11.52.13-ca-jre11.0.13-linux_i686.tar.gz" ;; # 32 Bit
*)     echo "ERROR: No java for $m" 2>&1; exit 1
esac

echo -n "* Downloading: jre ... "
wget -qO - "$jre" | tar xzf - && echo OK

echo -n "* Downloading detector ... "
wget -q "$detector" && echo OK

java=$(find . -name java -type f -executable| head -1)
if [ -z "$java" ]; then
    echo "java not found" >&2
    exit 1
fi

find_opt=(
    /
    \( -type d \( -fstype autofs -o -fstype fuse.sshfs -o -fstype nfs -o -fstype proc -o -fstype sshfs -o -fstype sysfs -o -fstype tmpfs \) -prune -o -type f \)  
    -not -path  \*/.snapshots/\*
    -not -path  \*/.m2/repo/\*
    -type f -print
)

echo "* Scanning using $java and ${detector##*/}:"

while read line; do

    case "$line" in
    "-- Problem"*" encrypted "*) ;;         # HIDE
    "-- Problem"*".zip.ZipException"*) ;;   # HIDE
    "-- Problem"*".io.EOFException"*) ;;    # HIDE
    "-- Problem"*"no magic number"*) ;;     # HIDE
    "-- Problem"*"not find ZIP magic"*);;   # HIDE
    "-- Problem"*"malformed") ;;            # HIDE
    "-- Problem"*"invalid distance"*) ;;    # HIDE
    "-- Problem"*) echo "  ${line#-}";;     # SHOW (unknown problems)
    "-- "*);;                               # HIDE
    *" _POTENTIALLY_SAFE_"*);;              # HIDE
    *" _OLD_");;                            # HIDE (for the moment)
    *) echo "  - $line" ;;                  # SHOW (the rest)
    esac
done < <(find "${find_opt[@]}" | "$java" -jar ${detector##*/} --stdin 2>&1 || true)

Which produces, for example, this output:

* Downloading: jre ... OK
* Downloading detector ... OK
* Scanning using ./zulu11.52.13-ca-jre11.0.13-linux_x64/bin/java and log4j-detector-2021.12.29.jar:
  - /root/log4j-samples/false-hits/exploded/2.17.1/org/apache/logging/log4j contains Log4J-2.x   >= 2.17.1 _SAFE_
  - /root/log4j-samples/false-hits/exploded/2.3.1/org/apache/logging/log4j contains Log4J-2.x   == 2.3.1 _OKAY_
  - /root/log4j-samples/false-hits/exploded/2.17.0/org/apache/logging/log4j contains Log4J-2.x   == 2.17.0 _OKAY_
  - /root/log4j-samples/false-hits/exploded/2.3.2/org/apache/logging/log4j contains Log4J-2.x   == 2.3.2 _SAFE_
  - /root/log4j-samples/false-hits/exploded/2.16.0/org/apache/logging/log4j contains Log4J-2.x   == 2.16.0 _OKAY_
  - /root/log4j-samples/false-hits/exploded/2.12.2/org/apache/logging/log4j contains Log4J-2.x   == 2.12.2 _OKAY_
  - /root/log4j-samples/false-hits/exploded/2.12.4/org/apache/logging/log4j contains Log4J-2.x   == 2.12.4 _SAFE_
  - /root/log4j-samples/false-hits/log4j-core-2.3.2.jar contains Log4J-2.x   == 2.3.2 _SAFE_
  - /root/log4j-samples/false-hits/log4j-core-2.12.2.jar contains Log4J-2.x   == 2.12.2 _OKAY_
  - /root/log4j-samples/false-hits/log4j-core-2.17.0.jar contains Log4J-2.x   == 2.17.0 _OKAY_
  - /root/log4j-samples/false-hits/log4j-core-2.16.0.jar contains Log4J-2.x   == 2.16.0 _OKAY_
  - /root/log4j-samples/false-hits/log4j-core-2.12.3.jar contains Log4J-2.x   == 2.12.3 _OKAY_
  - /root/log4j-samples/false-hits/log4j-core-2.12.4.jar contains Log4J-2.x   == 2.12.4 _SAFE_
  - /root/log4j-samples/false-hits/log4j-core-2.17.1.jar contains Log4J-2.x   >= 2.17.1 _SAFE_
  - /root/log4j-samples/false-hits/log4j-core-2.3.1.jar contains Log4J-2.x   == 2.3.1 _OKAY_
  - /root/log4j-samples/false-hits/log4j-core-2.15.0.jar contains Log4J-2.x   == 2.15.0 _OKAY_
  - /root/log4j-samples/true-hits/log4j-core-2.4.jar contains Log4J-2.x   >= 2.0-beta9 (< 2.10.0) _VULNERABLE_
  - /root/log4j-samples/true-hits/exploded/2.12.1/org/apache/logging/log4j contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
  - /root/log4j-samples/true-hits/log4j-core-2.4.1.jar contains Log4J-2.x   >= 2.0-beta9 (< 2.10.0) _VULNERABLE_
  - /root/log4j-samples/true-hits/log4j-core-2.12.0.jar contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
  - /root/log4j-samples/true-hits/log4j-core-2.14.1.jar contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
  - /root/log4j-samples/true-hits/uber/infinispan-embedded-query-8.2.12.Final.jar contains Log4J-2.x   >= 2.0-beta9 (< 2.10.0) _VULNERABLE_
  - /root/log4j-samples/true-hits/springboot-executable/spiff-0.0.1-SNAPSHOT.war!/WEB-INF/lib/log4j-core-2.10.0.jar contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
  - /root/log4j-samples/true-hits/springboot-executable/spiff-0.0.1-SNAPSHOT.jar!/WEB-INF/lib/log4j-core-2.10.0.jar contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
  - /root/log4j-samples/true-hits/springboot-executable/spiff-0.0.1-SNAPSHOT.zip!/WEB-INF/lib/log4j-core-2.10.0.jar contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
  - /root/log4j-samples/true-hits/springboot-executable/spiff-0.0.1-SNAPSHOT.ear!/WEB-INF/lib/log4j-core-2.10.0.jar contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
  - /root/log4j-samples/true-hits/log4j-core-2.3.jar contains Log4J-2.x   >= 2.0-beta9 (< 2.10.0) _VULNERABLE_
  - /root/log4j-samples/true-hits/log4j-core-2.11.2.jar contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
  - /root/log4j-samples/true-hits/log4j-core-2.11.1.jar contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
  - /root/log4j-samples/true-hits/log4j-core-2.2.jar contains Log4J-2.x   >= 2.0-beta9 (< 2.10.0) _VULNERABLE_
  - /root/log4j-samples/true-hits/log4j-core-2.0-beta9.jar contains Log4J-2.x   >= 2.0-beta9 (< 2.10.0) _VULNERABLE_
  - /root/log4j-samples/true-hits/log4j-core-2.9.1.jar contains Log4J-2.x   >= 2.0-beta9 (< 2.10.0) _VULNERABLE_
  - /root/log4j-samples/true-hits/shaded/clt-1.0-SNAPSHOT.jar contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
  - /root/log4j-samples/true-hits/log4j-core-2.12.1.jar contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
  - /root/log4j-samples/true-hits/log4j-core-2.14.0.jar contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
  - /root/log4j-samples/true-hits/log4j-core-2.10.0.zip contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
  - /root/log4j-samples/true-hits/log4j-core-2.11.0.jar contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
  - /root/log4j-samples/true-hits/log4j-core-2.10.0.jar contains Log4J-2.x   >= 2.10.0 _VULNERABLE_
* Removing /tmp/tmp.oK41fgbR9b

[beckerr-rzht]

@pthoelken

-- Skipping C:\home\username\log4j-samples\false-hits\log4j-api-2.14.1.jar - Not a regular file.

What surprises me: Where do the DOS paths come from? Is the script run under cygwin with java as Windows binary?

[pthoelken]

@pthoelken

-- Skipping C:\home\username\log4j-samples\false-hits\log4j-api-2.14.1.jar - Not a regular file.

What surprises me: Where do the DOS paths come from? Is the script run under cygwin with java as Windows binary?

Right. Currently I can test this at Windows (CYGWin) only. Thanks for your script. I will test this in the next few days.

[beckerr-rzht]

Right. Currently I can test this at Windows (CYGWin) only. Thanks for your script. I will test this in the next few days.

Have you considered using WSL? This way you could download a JRE on the fly (like I do) that matches the "find" execution layer. This would also have the benefit that you can be sure not to run into this bug: #69

[pthoelken]

Right. Currently I can test this at Windows (CYGWin) only. Thanks for your script. I will test this in the next few days.

Have you considered using WSL? This way you could download a JRE on the fly (like I do) that matches the "find" execution layer. This would also have the benefit that you can be sure not to run into this bug: #69

Yea, I know but at my business workstation wsl doesn't work correctly atm (VirtualBox, Hyper-V, Docker ... struggle) but I can test it in the evening on my home desk.

[beckerr-rzht]

What you also could try: The findutils are available in a variant for Windows (See http://gnuwin32.sourceforge.net/packages/findutils.htm). The output of find and the path syntax used by java should then be more compatible. I haven't tested this yet, but the example looks promising:

[SonamorN]

Why not use powershell and Get-ChildItem and end up using Linux stuff in Windows?

On Thu, Jan 6, 2022, 10:26 beckerr-rzht @.***> wrote:

What you also could try: The findutils are available in a variant for Windows (See http://gnuwin32.sourceforge.net/packages/findutils.htm). The output of find and the path syntax used by java should then be more compatible. I haven't tested this yet, but the example looks promising: [image: grafik] https://user-images.githubusercontent.com/15359213/148352406-6a533f26-f01a-4228-9d8b-5bc2a0517360.png

— Reply to this email directly, view it on GitHub https://github.com/mergebase/log4j-detector/pull/75#issuecomment-1006371036, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAI4X4KR7LDQWU445VFH2PTUUVG25ANCNFSM5LANS5LQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[beckerr-rzht]

Why not use powershell and Get-ChildItem and end up using Linux stuff in Windows?

You are right about that, of course. But the actual question and my workaround referred to an existing bash script: https://github.com/pthoelken/log4j-searcher/blob/55f263aad20ed29f36f3476cfebb1ee3c0524f3b/runner.sh#L58

[pthoelken]

For your explain: This script is not for windows environment. I've just code it on a windows computer because my macbook is still in delivery.

When I have to be code this for windows env, of course I choose ps1.

[SonamorN]

Ok then my bad. I was missing this context when I decided to reply.

On Thu, Jan 6, 2022, 11:10 Patrick Thoelken @.***> wrote:

For your explain: This script is not for windows environment. I've just code it on a windows computer because my macbook is still in delivery.

When I have to be code this for windows env, of course I choose ps1.

— Reply to this email directly, view it on GitHub https://github.com/mergebase/log4j-detector/pull/75#issuecomment-1006396950, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAI4X4IB5SXZ25NUU4S7RK3UUVL73ANCNFSM5LANS5LQ . You are receiving this because you commented.Message ID: @.***>

[pthoelken]

Ok then my bad. I was missing this context when I decided to reply. … On Thu, Jan 6, 2022, 11:10 Patrick Thoelken @.> wrote: For your explain: This script is not for windows environment. I've just code it on a windows computer because my macbook is still in delivery. When I have to be code this for windows env, of course I choose ps1. — Reply to this email directly, view it on GitHub <#75 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAI4X4IB5SXZ25NUU4S7RK3UUVL73ANCNFSM5LANS5LQ . You are receiving this because you commented.Message ID: @.>

No problem :)