nopperl
nopperl
I'm also interested in this. I've converted the model to GGML here: https://huggingface.co/nopperl/alpaca-lora-7b-german-base-51k-ggml. I can also add quantified models if necessary.
@schmorp I've fixed the conversion bug in #6910. >OLMo-7B-SFT fails differently: > >KeyError: "could not find any of: ['hidden_size', 'n_embd']" This is because the conversion script only supports OLMo in...
@JacobGoldenArt in the provided docker compose setup, the model is not stored in the container! Instead a host directory is mounted into the container. Also, exllama expects the directory to...
@vladmandic I have now removed `tcmalloc` and the changes to the `README.md`. Regarding `--data-dir`, I thought that using a subdir of the workdir of the container would be a sane...
Good idea, I have made it mandatory now
I built it again from scratch and noticed an error ^^ The `requirements.txt` file was ignored due to the `/*.txt` entry in the ignore file. Now it works.
@vladmandic I think its ready to be merged
Very unfortunate that the `--skip-update` flag was removed, thanks for bringing it to my attention @staff0rd. I think solving this indirectly by storing the packages and repositories in a bind-mounted...
@Kubuxu thanks for the suggestions, I've fixed the env vars. >I would also suggest making the first argument to the entrypoint webui and setting it by default with RUN ["webui"]...