Bug: When using pandas in my dependencies, all the extra_dependencies are checked even they are not installed

[12rambau]

Before You Begin

Before proceeding, please make sure to follow these steps:

• [x] I have checked for similar issues in the project's issue tracker.
• [x] I have searched closed issues to see if a similar problem was reported before.

Issue Details

I get pipeline crashes on various projects recently because of LicenseCheck upgrade to 2023.5.1 from 2023.1.1. whenever I check in pandas in my deps, all its extra dependencies are checked as well including PyQT5 that is under GPL3 which is incompatible with MIT.

My licensecheck parameters

[tool.licensecheck]
using = "PEP631"

the dependencies:

dependencies = [
    "deprecated>=1.2.14",
    "pandas<2",
    "earthengine-api",
    "pyarrow"
]

and the result from the pre-commit hook:

┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Item            ┃ Value        ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ program         │ licensecheck │
│ version         │ 2023.5.1     │
│ license         │ MIT LICENSE  │
│ project_license │ MIT LICENSE  │
└─────────────────┴──────────────┘
                                                                                
                                List Of Packages                                
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Compatible ┃ Package              ┃ License(s)                               ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ ✔          │ Bottleneck           │ BSD LICENSE                              │
│ ✔          │ Deprecated           │ MIT LICENSE                              │
│ ✔          │ Jinja2               │ BSD LICENSE                              │
│ ✖          │ PyQt5                │ GPL V3                                   │
│ ✔          │ QtPy                 │ MIT LICENSE                              │
│ ✔          │ SQLAlchemy           │ MIT LICENSE                              │
│ ✔          │ XlsxWriter           │ BSD LICENSE                              │
│ ✔          │ beautifulsoup4       │ MIT LICENSE                              │
│ ✔          │ dataframe-api-compat │ MIT LICENSE                              │
│ ✔          │ earthengine-api      │ APACHE SOFTWARE LICENSE                  │
│ ✔          │ fastparquet          │ APACHE SOFTWARE LICENSE                  │
│ ✔          │ fsspec               │ BSD LICENSE                              │
│ ✔          │ gcsfs                │ BSD LICENSE                              │
│ ✔          │ html5lib             │ MIT LICENSE                              │
│ ✔          │ hypothesis           │ MOZILLA PUBLIC LICENSE 2.0 (MPL 2.0)     │
│ ✔          │ lxml                 │ BSD LICENSE                              │
│ ✔          │ matplotlib           │ PYTHON SOFTWARE FOUNDATION LICENSE       │
│ ✔          │ numba                │ BSD LICENSE                              │
│ ✔          │ numexpr              │ MIT LICENSE                              │
│ ✔          │ numpy                │ BSD LICENSE                              │
│ ✔          │ odfpy                │ APACHE SOFTWARE LICENSE;; GNU GENERAL    │
│            │                      │ PUBLIC LICENSE (GPL);; GNU LIBRARY OR    │
│            │                      │ LESSER GENERAL PUBLIC LICENSE (LGPL)     │
│ ✔          │ openpyxl             │ MIT LICENSE                              │
│ ✔          │ pandas               │ BSD LICENSE                              │
│ ✔          │ pandas-gbq           │ BSD LICENSE                              │
│ ✔          │ psycopg2             │ GNU LIBRARY OR LESSER GENERAL PUBLIC     │
│            │                      │ LICENSE (LGPL)                           │
│ ✔          │ pyarrow              │ APACHE SOFTWARE LICENSE                  │
│ ✔          │ pymysql              │ MIT LICENSE                              │
│ ✔          │ pyreadstat           │ APACHE SOFTWARE LICENSE                  │
│ ✔          │ pytest               │ MIT LICENSE                              │
│ ✔          │ pytest-asyncio       │ APACHE SOFTWARE LICENSE                  │
│ ✔          │ pytest-xdist         │ MIT LICENSE                              │
│ ✔          │ python-dateutil      │ APACHE SOFTWARE LICENSE;; BSD LICENSE    │
│ ✔          │ pytz                 │ MIT LICENSE                              │
│ ✔          │ pyxlsb               │ GNU LESSER GENERAL PUBLIC LICENSE V3 OR  │
│            │                      │ LATER (LGPLV3+)                          │
│ ✔          │ s3fs                 │ BSD LICENSE                              │
│ ✔          │ scipy                │ BSD LICENSE                              │
│ ✔          │ tables               │ BSD LICENSE                              │
│ ✔          │ tabulate             │ MIT LICENSE                              │
│ ✔          │ tzdata               │ APACHE SOFTWARE LICENSE                  │
│ ✔          │ wrapt                │ BSD LICENSE                              │
│ ✔          │ xarray               │ APACHE SOFTWARE LICENSE                  │
│ ✔          │ xlrd                 │ BSD LICENSE                              │
│ ✔          │ zstandard            │ BSD LICENSE                              │
└────────────┴──────────────────────┴──────────────────────────────────────────┘

Even though a simple pip install pandas && pip show PyQT5 shows that it's not installed by default.

Note that I'm using pandas > 2 that have changed to extra_requires recently.

Expected Behavior

I would expect to not see it in the list as it's not installed

[flozzone]

Seeing this behavior too. It seems that some packages use the extra marker to signalize it is optional but licensecheck is not able to support markers. I've looked into the code a bit to get an impression what would be needed to get this fixed, but it seems that it would require some more refactoring in https://github.com/FHPythonUtils/LicenseCheck/blob/master/licensecheck/get_deps.py in order to be able to respect markers.

[FredHappyface]

Thank you for the investigation on this! The code base has grown somewhat organically and could do with some TLC when I get the time. If anyone want's to take any of this on then you are welcome to do so. Otherwise it's something I do hope to get around to soon™

:)

[FredHappyface]

The latest commit https://github.com/FHPythonUtils/LicenseCheck/commit/ba70665012b66d0a2497f68fdeec6c3389a45a37 adds a new resolver which should solve this issue. I need to work out how to enable this (my concern is adding too many options to the cli - many I've previously added are already frustrating to me )

Shortly I'll add some tests to confirm this has fixed your issue. Thanks :)