proposal: Adopt pip-compile to improve requirements handling

[tmc]

I propose adopting pip-tools as a way to more flexibly and reliably manage the contents in requirements.txt.

Current Situation

Right now the requirements are managed in a bit of a hard to use manner and this is likely a hinderance to folks attempting to work on native Mac support (See https://github.com/Stability-AI/generative-models/issues/154 and https://github.com/Stability-AI/generative-models/issues/184).

The codebase today does not have a mechanism to define and or test on different platforms, effectively leaving it CUDA-only. There is also not a straightforward way to define tests and requirements against different python or CUDA versions.

The dependencies are managed between the ./requirements/p2.txt (I have a proposed simplification at https://github.com/Stability-AI/generative-models/pull/193) and along the path to running at test as defined in https://github.com/Stability-AI/generative-models/blob/main/pyproject.toml#L45

Rationale Only specifying one pip-freeze produced requirements file has a number of downsides:

• It's often a post-hoc description of some system being in some working state, instead of being a source-controlled definition of requirements.
• Adopting pip-compile means that version resolution becomes a development-time step instead of a build or installation time step.
• pip-compile grants us an easier to manage and more reproducible description format while enabling an easy path to targeting multi-platform support.

Possible Downsides and Tradeoffs

• pip-compile is another tool folks have to learn or cope with, this might mitigated by documentation and scripting/tooling to assist.

Alternatives Considered

• Current approach.
• Adopting poetry.