Packages used in codebase reported as `Extra requirements`

[db-baron]

I installed pip-check-reqs 2.1.1 and ran pip-extra-reqs on a django project to check for unused packages in the project's requirements.txt. The command output listed multiple packages as "Extra requirements", however several of these were incorrect as they are definitely used in the project codebase, including django, pandas, and plotly. On the other hand, several packages in my requirements.txt not used in the codebase were correctly listed under "Extra requirements". I asked a colleague to replicate these steps and they reported the same issue. Both of us double checked that we were running the command on the project's root directory.

[adamtheturtle]

Hi @db-baron thank you for the report. Do you have a codebase to share which I can reproduce this in? If not, do you think that you could create a small reproducible case?

[oleshynk]

@adamtheturtle I have the same situation with marking used packages as "Extra requirements". After running tests and debugging I figured out that a lot of installed packages are considered standard libraries(so shouldn't be in requirements). It happens because I'm running it on a Windows machine where paths usually have the next appearance "\\lib\\site-packages\\werkzeug\\wrappers\\__init__.py" while in Linux case it would be "/lib/site-packages/werkzeug/wrappers/__init__.py". Hence in Windows case it does not match the regex which adds installed package path to "installed_files"

Here it will match only a Linux-like path, but for Windows will always return '':

def is_package_file(path):
    '''Determines whether the path points to a Python package sentinel
    file - the __init__.py or its compiled variants.
    '''
    m = re.search(r'(.+)/__init__\.py[co]?$', path)
    if m is not None:
        return m.group(1)
    return ''

[theahura]

This may be a different bug, but figured I'd revive this thread.

In my case, it seems like pip-check-reqs is returning every package (used or not) as an extra requirement. Specifically, running

cat requirements.txt | wc -l returns the same number as pip-extra-reqs src &> extras.txt && cat extras.txt | wc -l

[MrMino]

This could be an issue with Windows (thanks @oleshynk for investigating this), or #81. In any case, unless a reproducible example is provided, it's impossible to fix it.

I will investigate the Windows issue further as soon as I figure out how to integrate another OS into my workflow without dual booting. After that I will close this issue, unless a reproducible example surfaces.

For the rest of the cases: please attach reproducible examples, otherwise we're unable to help.