Setup package

[paolorechia]

Supports installing exllama as a package.

Example usage:

 pip install 'exllama_lib @ git+https://github.com/paolorechia/exllama@setup-package'

EDIT: Worth explaining how to use the installed package. Since the installation setup creates a temporary exllama_lib to package exllama, the package also has this name, and can be imported as following:

from exllama_lib.model import ExLlama, ExLlamaCache, ExLlamaConfig
from exllama_lib.tokenizer import ExLlamaTokenizer
from exllama_lib.generator import ExLlamaGenerator

[paolorechia]

Thanks, @nikshepsvn nikshepsvn! I fixed it, and also squashed the commits in this branch since they were quite dirty.

[paolorechia]

@nopperl thanks for the review, I've updated the PR! Sorry for the delay, I wasn't working much in my free time lately.

[Dessix]

@turboderp There's now a reproducible Nix derivation that produces an installation of EXLlama, which includes a setup.py and compiles the CUDA extension on installation instead of at runtime- with or without a GPU on the system at the time of installation. Perhaps this will be of interest? It would certainly benefit from this PR, but may also help with automated testing or iteration of development.

[paolorechia]

Sorry, @Dessix, but I’m not planning to apply the requested changes. I appreciate the review and understand the raised issues but they go beyond the original intent of this PR.

Feel free to propose a different PR to replace this one.

[paolorechia]

Closing since it's stale.