cve-bin-tool

cve-bin-tool copied to clipboard

• related #1552

@Molkree discovered an interesting issue while improving our CI files:

Alright, it didn't take long, looks like my first guess was the correct one.

Two tests were failing in long tests job:

FAILED test/test_checkers.py::TestCheckerVersionParser::test_filename_is[curl-libcurl.so.4-expected_results3]
FAILED test/test_checkers.py::TestCheckerVersionParser::test_filename_is[bzip2-bzip2.so-expected_results1]

How it was before:

      # steps installing dependencies thru apt-get and pip
      - name: try single cli run of tool  # O1
        run: |
          python -m pip install -e .
          NO_EXIT_CVE_NUM=1 python -m cve_bin_tool.cli test/assets -n json -u latest
      - name: Run async tests  # O2
        run: >
          pytest --cov --cov-append -n 4 -v
          # skipping some tests here

How I made it:

      # steps installing dependencies thru apt-get
      - name: Install pdftotext and cve-bin-tool  # N1
        run: |
          python -m pip install --upgrade pip
          python -m pip install --upgrade setuptools
          python -m pip install --upgrade wheel
          python -m pip install --upgrade pdftotext
          python -m pip install --upgrade .
      - name: Try single CLI run of tool  # N2
        run: |
          NO_EXIT_CVE_NUM=1 python -m cve_bin_tool.cli test/assets -n json -u latest
      - name: Run async tests  # N3
        run: >
          pytest --cov --cov-append -n 4 -v
          # skipping some tests here

Albeit some rearranging of steps, the difference is in how I install cve-bin-tool itself. Previously it was an editable install (pip install -e . in step O1) and I changed it to a simple local install (pip install --upgrade . in step N1).

I don't understand why it breaks these tests. I don't understand why it only fails on 2 files out of 52 in this test. And finally, I don't understand why it doesn't fail in normal (not long) and Windows tests (where I've also replaced --editable with --upgrade). Any ideas are highly appreciated!

We've seen other issues with --editable (see #1323 ) so there's definitely something different happening there. if I had to guess, it might be related to the way editable uses a symlink to an existing directory but --upgrade should only copy over the files recommended by setup.py. But that's pure speculation at this point.