Pip/Pypi Python Package Distribution

[ptheywood]

To ease installation of pyflamegpu we likely want to provide one or more pypi package(s), so that users may install the latest pyflamegpu, using the default CUDA for that version via:

python3 -m pip install pyflamegpu

There are several outstanding issues before this would become viable:

• [ ] Manywheel2014 complaince #647
• [ ] Do we provide one or multiple packages?
  • pyflamegpu, pyflamegpu-vis, maybe pyflamegpu[visualiastion] is viable? Pip/pypi is not ideal for this. It may also be solvable via dlopen at the cost of filesize.
• [ ] Automation of package upload
  • This could be handled by an action on publicaiton of a release.
• [ ] Which CUDA version and architectures should be supported by the default build?
  • CUDA 11.2 would be the best choice as this will work with any CUDA 11 version greater than 11.2. Alternate versions can be handeled separately.
  • SM 35 is a sensible but old current minimum. We can build multi-arch wheels for improved performance on more recent architectures, but the more we include the fatter the binary.
• [ ] What about SEATBELTS=OFF and debug builds?
• --pre will be required until we have a stable release, so probably not a hurry to do this, although we may wish to reserve the package name.

We can provide our own almost index which users can opt in to for alternative builds that leverage the local version number in some cases, as done by pytorch. See #645 for this mostly independent task.

See #605 for previous disscussion notes on this.

[ptheywood]

We may not be able to achieve manylinux compliance for visualisation builds, so pypi may only be able to host console only (linux) builds.

[ptheywood]

~As a minimum we should reserve the package name on pypi to avoid supply chain hijacking~

This may not actually be advisable, as it might count as name squatting in PEP541. There are instances of people being asked to email the pypi admins about reserving package names short term source

[Robadob]

Considering this as "set up wheelhouse" and perhaps ask pypi to reserve package name

[ptheywood]

Considering this as "set up wheelhouse" and perhaps ask pypi to reserve package name

https://github.com/FLAMEGPU/FLAMEGPU2/issues/645 is the existing issue for that, this issue is for pypi based distribution.

[Robadob]

this issue is for pypi based distribution.

Ah, I had assumed PyPi was considered off the cards due to our size/libs (in which case this should be closed as wontfix) and missed the other issue.

[ptheywood]

Size is potentially resolved by asking nicely.

manylinux compliance / dlopen reqs would still be a good addition to the wheelhouse based pip installs, as it improves portability of the generated wheels, but is a lot of effort that is probably better spent elsewhere.

Either way, the above comments about package name squatting / supply chain hijacking should be conisdered as a minimum, as otherwise if someone does pip install pyflamegpu, they might get another package if someone makes one. However this could be considered namesquatting, but pypi have allowed this in the past iirc.

[ptheywood]

@mondus contacted pypi re: preventing supply chain hijacking and has had the name pyflamegpu blocked? not 100% on the details.

Arguably we could now close this as won't fix, though in an ideal world we would have a pypi compliant (read manylinux) build, and would have to add a single build (latest cuda, console only to simplify manylinuxing) as local version numbers can't be included, so we can't dist mutliple versions. Our wheelhouse (whl.flamegpu.com) can then be used to provide alt versions via pip.

Conda should probably still be the "preferred" python distribution though.