package with poetry to enable easy packaging for linux distribution

[happysalada]

Description

Using poetry for packaging would make builds reproducible and create a lockfile. This would enable easy packaging for linux distros. Particularly, I came here wanting to package this for nixos. If you used poetry it would be relatively easy to do. If you are open to the idea, I would be happy to have a look at how to make the switch. It shouldn't be too hard.

Additional Context

If applicable, please provide any extra information, external links, or screenshots that could be useful.

[Reezlaw]

Isn't the docker-compose method perfectly reproducible?

[happysalada]

Package resolution with normal python is not deterministic. Depending on when you install the requirements you would have different versions of the subpackages. Poetry has a lock file that defines exactly which subpackages version it's using. To make sure that any two installs are exactly the same. Docker is just an abstraction on top of that, so if you build a docker image let's say 2 weeks from now, you'll probably get different sha256 for each layer because they will be different. When you make a package for a linux distro, you usually package the binary and not the docker image. If you want to make this package available to other linux distros with their package manager, having a distributable binary is a must. Last but not least, I wanted to propose poetry as it didn't seem to add any burden to the maintainers, if you feel it's too much work, no worries.

[Reezlaw]

Oh no, I'm just a random schmuck, I was genuinely asking. Maybe this is a great idea

[happysalada]

Not at all, clarifying is always good !

[github-actions[bot]]

This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.

[khimaros]

i've produced a .tool-versions (used with rtx) and pyproject.toml which is working well for CPU only inference.

.tool-versions

python 3.10.6

pyproject.toml

[tool.poetry]
name = "text-generation-webui"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"

[[tool.poetry.source]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
priority = "explicit"

[tool.poetry.dependencies]
python = ">=3.10,<3.12"
accelerate = {git = "https://github.com/huggingface/accelerate", rev = "0226f750257b3bf2cadc4f189f9eef0c764a0467"}
llama-cpp-python = "0.1.53"
bitsandbytes = "0.39.0"
transformers = {git = "https://github.com/huggingface/transformers", rev = "e45e756d22206ca8fa9fb057c8c3d8fa79bf81c6"}
peft = {git = "https://github.com/huggingface/peft", rev = "3714aa2fff158fdfa637b2b65952580801d890b2"}
tqdm = "^4.65.0"
sentencepiece = "^0.1.99"
safetensors = "0.3.1"
rwkv = "0.7.3"
requests = "^2.30.0"
pyyaml = "^6.0"
pillow = ">=9.5.0"
pandas = "^2.0.1"
numpy = "^1.24.3"
markdown = "^3.4.3"
gradio = "3.31.0"
flexgen = "0.1.7"
datasets = "^2.12.0"
colorama = "^0.4.6"
einops = "^0.6.1"
gradio-client = "0.2.5"
scipy = "^1.10.1"
torch = {version = "^2.0.1", source = "pytorch-cpu"}
torchvision = {version = "^0.15.2", source = "pytorch-cpu"}
torchaudio = {version = "^2.0.2", source = "pytorch-cpu"}

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

to install dependencies in a venv and start the server:

pip install poetry

poetry install --sync

poetry run server.py --cpu

[ParetoOptimalDev]

Isn't the docker-compose method perfectly reproducible?

Docker adds in a layer of complexity in getting nvidia-container-toolkit to work for gpu accelleration for instance. Ask me how I know :stuck_out_tongue_winking_eye:

[khimaros]

the python ecosystem seems to be moving toward pip-tools instead of poetry. mixes well with rtx

[Reezlaw]

Isn't the docker-compose method perfectly reproducible?

Docker adds in a layer of complexity in getting nvidia-container-toolkit to work for gpu accelleration for instance. Ask me how I know 😜

Really? That was never a problem for me