feat: Add extraction support for .bin files (Fixes #1821)

Open anthonyharrison opened this issue 2 years ago • 7 comments

Codecov Report

Merging #1850 (623f8d5) into main (1bd8652) will increase coverage by 8.56%. The diff coverage is 58.82%.

@@            Coverage Diff             @@
##             main    #1850      +/-   ##
==========================================
+ Coverage   78.43%   87.00%   +8.56%     
==========================================
  Files         314      318       +4     
  Lines        7109     7302     +193     
  Branches     1160     1192      +32     
==========================================
+ Hits         5576     6353     +777     
+ Misses       1297      679     -618     
- Partials      236      270      +34

Flag	Coverage Δ
longtests	`78.12% <58.82%> (-0.31%)`	:arrow_down:
win-longtests	`85.92% <47.05%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
cve_bin_tool/extractor.py	`80.82% <27.27%> (+10.14%)`	:arrow_up:
test/test_extractor.py	`91.21% <70.58%> (-2.12%)`	:arrow_down:
test/utils.py	`92.85% <83.33%> (-1.59%)`	:arrow_down:
cve_bin_tool/sbom_manager/cyclonedx_parser.py	`80.43% <0.00%> (-4.35%)`	:arrow_down:
cve_bin_tool/parsers/javascript.py	`87.87% <0.00%> (-3.96%)`	:arrow_down:
cve_bin_tool/output_engine/html.py	`88.46% <0.00%> (-3.08%)`	:arrow_down:
cve_bin_tool/version_scanner.py	`85.03% <0.00%> (-0.98%)`	:arrow_down:
cve_bin_tool/parsers/python.py	`85.71% <0.00%> (-0.78%)`	:arrow_down:
cve_bin_tool/cve_scanner.py	`85.43% <0.00%> (-0.10%)`	:arrow_down:
... and 40 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Jul 31 '22 19:07 codecov-commenter

I'm going to update this branch for the OSV fixes.

Aug 01 '22 17:08 terriko

Something is going horribly wrong here. Why does it think so many checker names are invalid?

------------------------------ Captured log call -------------------------------
INFO     cve_bin_tool:cli.py:406 CVE Binary Tool v3.1.1
INFO     cve_bin_tool:cli.py:407 This product uses the NVD API but is not endorsed or certified by the NVD.
INFO     cve_bin_tool:cli.py:421 Not using an NVD API key. Your access may be rate limited by NVD.
INFO     cve_bin_tool:cli.py:422 Get an NVD API key here: https://nvd.nist.gov/developers/request-an-api-key
INFO     cve_bin_tool.CVEDB:cvedb.py:138 Using cached CVE data (<24h old). Use -u now to update immediately.
INFO     cve_bin_tool.CVEDB:cvedb.py:168 There are 182079 CVE entries in the database
INFO     cve_bin_tool:cli.py:537 CVE database last updated on 01 August 2022 at 17:44:54
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker accountsservice is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker avahi is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker bash is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker bind is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker bolt is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker bubblewrap is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker busybox is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker bzip2 is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker commons_compress is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker cronie is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker cryptsetup is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker cups is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker dbus is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker dnsmasq is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker dovecot is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker dpkg is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker enscript is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker expat is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker ffmpeg is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker freeradius is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker ftp is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gcc is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gimp is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker glibc is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gnomeshell is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gnupg is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gnutls is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gpgme is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gstreamer is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gupnp is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker haproxy is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker hdf5 is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker hostapd is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker hunspell is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker icecast is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker icu is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker irssi is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker jacksondatabind is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker kbd is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker kerberos is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker kexectools is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libarchive is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libbpg is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libdb is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libebml is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libgcrypt is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libical is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libjpeg_turbo is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker liblas is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libnss is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker librsvg is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libseccomp is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libsndfile is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libsolv is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libsoup is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libsrtp is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libssh2 is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libtiff is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libvirt is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libvncserver is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libxslt is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker lighttpd is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker logrotate is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker lua is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker luajit is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker mariadb is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker mdadm is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker memcached is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker mtr is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker mysql is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker nano is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker ncurses is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker nessus is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker netpbm is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker nginx is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker node is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker ntp is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker open_vm_tools is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker openafs is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker openjpeg is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker openldap is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker openssh is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker openssl is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker openswan is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker openvpn is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker p7zip is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker pcsc_lite is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker pigz is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker png is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker polarssl_fedora is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker poppler is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker postgresql is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker pspp is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker python is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker qt is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker radare2 is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker rsyslog is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker rust is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker samba is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker sane_backends is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker sqlite is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker strongswan is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker subversion is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker sudo is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker syslogng is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker systemd is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker tcpdump is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker trousers is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker varnish is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker webkitgtk is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker wireshark is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker wpa_supplicant is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker xerces is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker xml2 is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker zlib is not a valid checker name
ERROR    cve_bin_tool.VersionScanner:version_scanner.py:98 Checker zsh is not a valid checker name
INFO     cve_bin_tool:cli.py:677 Number of checkers: 2
INFO     cve_bin_tool.VersionScanner:version_scanner.py:101 Checkers: binutils, curl
INFO     cve_bin_tool.CVEScanner:cve_scanner.py:238 58 CVE(s) in haxx.curl v7.34.0
INFO     cve_bin_tool.CVEScanner:cve_scanner.py:238 30 CVE(s) in haxx.libcurl v7.34.0
WARNING  cve_bin_tool.VersionScanner:extractor.py:345 Failure extracting /home/runner/work/cve-bin-tool/cve-bin-tool/test/assets/test.bin
INFO     cve_bin_tool.CVEScanner:cve_scanner.py:238 26 CVE(s) in haxx.curl v7.66.0
INFO     cve_bin_tool.CVEScanner:cve_scanner.py:238 6 CVE(s) in haxx.libcurl v7.66.0
INFO     cve_bin_tool:cli.py:711 
INFO     cve_bin_tool:cli.py:712 Overall CVE summary: 
INFO     cve_bin_tool:cli.py:714 There are 4 products with known CVEs detected
INFO     cve_bin_tool:cli.py:728 Known CVEs in ('curl', '7.34.0'), ('curl', '7.66.0'), ('libcurl', '7.34.0'), ('libcurl', '7.66.0'):
=============================== warnings summary ===============================
<frozen importlib._bootstrap>:283
  <frozen importlib._bootstrap>:283: DeprecationWarning: the load_module() method is deprecated and slated for removal in Python 3.12; use exec_module() instead

test/test_cli.py: 26 warnings
  /home/runner/work/cve-bin-tool/cve-bin-tool/cve_bin_tool/version_scanner.py:85: DeprecationWarning: SelectableGroups dict interface is deprecated. Use select.
    checkers = importlib_metadata.entry_points()[cls.CHECKER_ENTRYPOINT]

test/test_cli.py: 15 warnings
  /home/runner/work/cve-bin-tool/cve-bin-tool/cve_bin_tool/version_scanner.py:78: DeprecationWarning: SelectableGroups dict interface is deprecated. Use select.
    importlib_metadata.entry_points()[cls.CHECKER_ENTRYPOINT],

test/test_cli.py::TestCLI::test_runs
test/test_cli.py::TestCLI::test_runs
test/test_cli.py::TestCLI::test_config_file[/home/runner/work/cve-bin-tool/cve-bin-tool/test/config/cve_bin_tool_config.toml]
test/test_cli.py::TestCLI::test_config_file[/home/runner/work/cve-bin-tool/cve-bin-tool/test/config/cve_bin_tool_config.yaml]
  /home/runner/work/cve-bin-tool/cve-bin-tool/cve_bin_tool/cli.py:[612](https://github.com/intel/cve-bin-tool/runs/7616470528?check_suite_focus=true#step:12:613): DeprecationWarning: SelectableGroups dict interface is deprecated. Use select.
    importlib_metadata.entry_points()["cve_bin_tool.checker"],

test/test_cli.py: 20 warnings
  /opt/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/site-packages/packaging/version.py:111: DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test/test_cli.py::TestCLI::test_config_file[/home/runner/work/cve-bin-tool/cve-bin-tool/test/config/cve_bin_tool_config.toml]
FAILED test/test_cli.py::TestCLI::test_config_file[/home/runner/work/cve-bin-tool/cve-bin-tool/test/config/cve_bin_tool_config.yaml]
============ 2 failed, 12 passed, 8 skipped, 66 warnings in 40.87s =============

Aug 01 '22 18:08 terriko

I can't see how this code could have possibly have caused the errors I'm seeing, so I'm going to try re-running CI just to see if it's a consistent issue or some weirdness in github actions. Relevant log is pasted above for posterity.

Aug 01 '22 18:08 terriko

I have installed the binwalk application using apt-get install. It drags in a lot of dependencies - I wonder if there is a clash or version mismatch...

On Mon, 1 Aug 2022, 19:17 Terri Oda, @.***> wrote:

Something is going horribly wrong here. Why does it think so many checker names are invalid?

------------------------------ Captured log call ------------------------------- INFO cve_bin_tool:cli.py:406 CVE Binary Tool v3.1.1 INFO cve_bin_tool:cli.py:407 This product uses the NVD API but is not endorsed or certified by the NVD. INFO cve_bin_tool:cli.py:421 Not using an NVD API key. Your access may be rate limited by NVD. INFO cve_bin_tool:cli.py:422 Get an NVD API key here: https://nvd.nist.gov/developers/request-an-api-key INFO cve_bin_tool.CVEDB:cvedb.py:138 Using cached CVE data (<24h old). Use -u now to update immediately. INFO cve_bin_tool.CVEDB:cvedb.py:168 There are 182079 CVE entries in the database INFO cve_bin_tool:cli.py:537 CVE database last updated on 01 August 2022 at 17:44:54 ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker accountsservice is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker avahi is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker bash is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker bind is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker bolt is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker bubblewrap is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker busybox is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker bzip2 is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker commons_compress is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker cronie is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker cryptsetup is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker cups is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker dbus is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker dnsmasq is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker dovecot is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker dpkg is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker enscript is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker expat is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker ffmpeg is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker freeradius is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker ftp is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gcc is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gimp is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker glibc is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gnomeshell is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gnupg is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gnutls is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gpgme is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gstreamer is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker gupnp is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker haproxy is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker hdf5 is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker hostapd is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker hunspell is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker icecast is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker icu is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker irssi is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker jacksondatabind is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker kbd is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker kerberos is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker kexectools is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libarchive is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libbpg is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libdb is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libebml is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libgcrypt is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libical is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libjpeg_turbo is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker liblas is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libnss is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker librsvg is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libseccomp is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libsndfile is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libsolv is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libsoup is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libsrtp is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libssh2 is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libtiff is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libvirt is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libvncserver is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker libxslt is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker lighttpd is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker logrotate is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker lua is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker luajit is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker mariadb is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker mdadm is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker memcached is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker mtr is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker mysql is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker nano is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker ncurses is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker nessus is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker netpbm is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker nginx is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker node is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker ntp is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker open_vm_tools is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker openafs is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker openjpeg is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker openldap is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker openssh is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker openssl is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker openswan is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker openvpn is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker p7zip is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker pcsc_lite is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker pigz is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker png is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker polarssl_fedora is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker poppler is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker postgresql is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker pspp is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker python is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker qt is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker radare2 is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker rsyslog is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker rust is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker samba is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker sane_backends is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker sqlite is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker strongswan is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker subversion is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker sudo is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker syslogng is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker systemd is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker tcpdump is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker trousers is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker varnish is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker webkitgtk is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker wireshark is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker wpa_supplicant is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker xerces is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker xml2 is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker zlib is not a valid checker name ERROR cve_bin_tool.VersionScanner:version_scanner.py:98 Checker zsh is not a valid checker name INFO cve_bin_tool:cli.py:677 Number of checkers: 2 INFO cve_bin_tool.VersionScanner:version_scanner.py:101 Checkers: binutils, curl INFO cve_bin_tool.CVEScanner:cve_scanner.py:238 58 CVE(s) in haxx.curl v7.34.0 INFO cve_bin_tool.CVEScanner:cve_scanner.py:238 30 CVE(s) in haxx.libcurl v7.34.0 WARNING cve_bin_tool.VersionScanner:extractor.py:345 Failure extracting /home/runner/work/cve-bin-tool/cve-bin-tool/test/assets/test.bin INFO cve_bin_tool.CVEScanner:cve_scanner.py:238 26 CVE(s) in haxx.curl v7.66.0 INFO cve_bin_tool.CVEScanner:cve_scanner.py:238 6 CVE(s) in haxx.libcurl v7.66.0 INFO cve_bin_tool:cli.py:711 INFO cve_bin_tool:cli.py:712 Overall CVE summary: INFO cve_bin_tool:cli.py:714 There are 4 products with known CVEs detected INFO cve_bin_tool:cli.py:728 Known CVEs in ('curl', '7.34.0'), ('curl', '7.66.0'), ('libcurl', '7.34.0'), ('libcurl', '7.66.0'): =============================== warnings summary =============================== :283 :283: DeprecationWarning: the load_module() method is deprecated and slated for removal in Python 3.12; use exec_module() instead

test/test_cli.py: 26 warnings /home/runner/work/cve-bin-tool/cve-bin-tool/cve_bin_tool/version_scanner.py:85: DeprecationWarning: SelectableGroups dict interface is deprecated. Use select. checkers = importlib_metadata.entry_points()[cls.CHECKER_ENTRYPOINT]

test/test_cli.py: 15 warnings /home/runner/work/cve-bin-tool/cve-bin-tool/cve_bin_tool/version_scanner.py:78: DeprecationWarning: SelectableGroups dict interface is deprecated. Use select. importlib_metadata.entry_points()[cls.CHECKER_ENTRYPOINT],

test/test_cli.py::TestCLI::test_runs test/test_cli.py::TestCLI::test_runs test/test_cli.py::TestCLI::test_config_file[/home/runner/work/cve-bin-tool/cve-bin-tool/test/config/cve_bin_tool_config.toml] test/test_cli.py::TestCLI::test_config_file[/home/runner/work/cve-bin-tool/cve-bin-tool/test/config/cve_bin_tool_config.yaml] /home/runner/work/cve-bin-tool/cve-bin-tool/cve_bin_tool/cli.py:612: DeprecationWarning: SelectableGroups dict interface is deprecated. Use select. importlib_metadata.entry_points()["cve_bin_tool.checker"],

test/test_cli.py: 20 warnings /opt/hostedtoolcache/Python/3.10.5/x64/lib/python3.10/site-packages/packaging/version.py:111: DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================== short test summary info ============================ FAILED test/test_cli.py::TestCLI::test_config_file[/home/runner/work/cve-bin-tool/cve-bin-tool/test/config/cve_bin_tool_config.toml] FAILED test/test_cli.py::TestCLI::test_config_file[/home/runner/work/cve-bin-tool/cve-bin-tool/test/config/cve_bin_tool_config.yaml] ============ 2 failed, 12 passed, 8 skipped, 66 warnings in 40.87s =============

— Reply to this email directly, view it on GitHub https://github.com/intel/cve-bin-tool/pull/1850#issuecomment-1201548179, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACAID276EBHXCGUMESY3WXTVXAIDZANCNFSM55FHY3LQ . You are receiving this because you authored the thread.Message ID: @.***>

Aug 01 '22 18:08 anthonyharrison

@terriko Some progress (but no solution yet).

Tried 3.7 without installing binwalk. No issues with the checkers but testing failed in test_extractor. Tried 3.8 with binwalk installed. Get the same issues with checkers not found Tried 3.10 without installing binwalk. Everything worked but new .bin tests failed because binwalk wasn't found

So it looks like there imight be an issue with some of the dependencies which are installed with binwalk which are impacting the checkers.

Aug 05 '22 15:08 anthonyharrison

@terriko No idea what is going on here! I haven't been anywhere near the NVD Schema..

______________________ ERROR collecting test/test_json.py ______________________
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/requests/models.py:971: in json
    return complexjson.loads(self.text, **kwargs)
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/json/__init__.py:346: in loads
    return _default_decoder.decode(s)
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/json/decoder.py:337: in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/json/decoder.py:355: in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
E   json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:
test/test_json.py:28: in <module>
    class TestJSON:
test/test_json.py:30: in TestJSON
    SCHEMA = requests.get(NVD_SCHEMA).json()
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/requests/models.py:975: in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)

Looks like there has been a glitch with NVD schema download. Puzzled why (a) the tests are being run (should only be on Python 3.8 i.e. Long Tests) and (b) why it isn't reporting which year is being validated.

Aug 26 '22 11:08 anthonyharrison

Hey @anthonyharrison I'm cleaning up PRs in preparation for hacktoberfest. I've marked this one as "blocked" so I won't keep revisiting it, but I'm drawing a blank on where we were with it. Did you want to close it and revisit when we have a better idea of what's going wrong, or do you think we could maybe separate out the tests to resolve the environment problem?

You know, now that you've got the sbom stuff set up... maybe we should install regular cve-bin-tool and then this PR and see if we can narrow down what package might be interfering with the parsing?

Sep 27 '22 23:09 terriko

I'm cleaning up old pull requests in preparation for the hackathon this month. I don't think any of us know how to resolve the dependency issues here, so I'm going to go ahead and close it for now.

Apr 17 '23 20:04 terriko

cve-bin-tool cve-bin-tool copied to clipboard

feat: Add extraction support for .bin files (Fixes #1821)

Codecov Report

cve-bin-tool
cve-bin-tool copied to clipboard