bandersnatch icon indicating copy to clipboard operation
bandersnatch copied to clipboard

Add platform tags specified in PEP 599/600 to exclude_platform

Open rigelifland opened this issue 4 years ago • 4 comments

Currently, if you use the exclude_platform filter to remove linux packages, newer package tags still make it through. (For example, tornado-6.1-cp35-cp35m-manylinux2014_aarch64.whl)

The exclude_platform filter currently doesn't exclude packages with newer linux platform tags as defined in PEP 599 and 600. Can these be added?

rigelifland avatar Feb 19 '21 19:02 rigelifland

PEP 599 should be straight forward to implement:

bandersnatch_filter_plugins/filename_name.py

    _linuxPlatformTypes = [
        "linux-i686",  # PEP 425
        "linux-x86_64",  # PEP 425
        "linux_armv7l",  # https://github.com/pypa/warehouse/pull/2010
        "linux_armv6l",  # https://github.com/pypa/warehouse/pull/2012
        "manylinux1_i686",  # PEP 513
        "manylinux1_x86_64",  # PEP 513
        "manylinux2010_i686",  # PEP 571
        "manylinux2010_x86_64",  # PEP 571
        "manylinux2014_x86_64",  # PEP 599
        "manylinux2014_i686",  # PEP 599
        "manylinux2014_aarch64",  # PEP 599
        "manylinux2014_armv7l",  # PEP 599
        "manylinux2014_ppc64",  # PEP 599
        "manylinux2014_ppc64le",  # PEP 599
        "manylinux2014_s390x"  # PEP 599
    ]

rigelifland avatar Feb 19 '21 19:02 rigelifland

Sure PR will be accepted for sure.

Long term tho, is there a library or should we look at adding this to the packaging library (https://github.com/pypa/packaging) if it's not already there as I am sure multiple libraries need this information.

cooperlees avatar Feb 19 '21 20:02 cooperlees

As far as I can tell, the packaging library is aware of PEP 599/600 but it doesn't have the ability to generate a list of tags which could be used as a replacement for _linuxPlatformTypes.

Some of the this problem stems from platforms in bandersnatch being simplified to just ['windows', 'linux', macos', 'freebsd'], where platforms in packaging includes architecture information (e.g.'win_amd64'). PEP 600 will exacerbate this with many new linux platform types - does it make sense to change from the current filter using in:

if i in fn:

to something more flexible like a regex match against the platform as parsed out by packaging:

if re.match(i, fn):

rigelifland avatar Feb 19 '21 23:02 rigelifland

Sure, happy to improve this generic matching as I think it's nice to say "ignore all windows" if you're a Linux only shop etc.

Thanks for this work!

cooperlees avatar Feb 20 '21 16:02 cooperlees