model_backend argument doesn't handle spaces (even properly quoted)

[BlairSadewitz]

[email protected]:~/KoboldAI$ ./play.sh --model models/Aurora-Nights-103B-v1.0-5.0bpw-h6-exl2 --model_backend "ExLlama V2" --model_parameters help Colab Check: False, TPU: False INFO | main::732 - We loaded the following model backends: KoboldAI API KoboldAI Old Colab Method Basic Huggingface ExLlama ExLlama V2 Huggingface GooseAI Legacy GPTQ Horde KoboldCPP OpenAI Read Only INFO | main:general_startup:1447 - Running on Repo: http://github.com/henk717/KoboldAI.git Branch: united usage: aiserver.py [-h] [--remote] [--noaimenu] [--ngrok] [--localtunnel] [--host [HOST]] [--port PORT] [--aria2_port ARIA2_PORT] [--model MODEL] [--model_backend MODEL_BACKEND] [--model_parameters MODEL_PARAMETERS] [--path PATH] [--apikey APIKEY] [--sh_apikey SH_APIKEY] [--req_model REQ_MODEL] [--revision REVISION] [--cpu] [--override_delete] [--override_rename] [--configname CONFIGNAME] [--colab] [--nobreakmodel] [--unblock] [--quiet] [--no_aria2] [--lowmem] [--savemodel] [--cacheonly] [--customsettings CUSTOMSETTINGS] [--no_ui] [--summarizer_model SUMMARIZER_MODEL] [--max_summary_length MAX_SUMMARY_LENGTH] [--multi_story] [--peft PEFT] [-f F] [-v] [-q] [--panic] aiserver.py: error: unrecognized arguments: V2

--

Am I doing something wrong here? I can escape the space with the shell, too, and it still breaks. Works fine if I remove the space in the assignment of model_backend_name in exllamav2/class.py.

[Belarrius1]

I have the same problem, you can modify your "play.sh" with

#!/bin/bash
export PYTHONNOUSERSITE=1
if [ ! -f "runtime/envs/koboldai/bin/python" ]; then
./install_requirements.sh cuda
fi
bin/micromamba run -r runtime -n koboldai python aiserver.py "$@"

Then you can lunch with --model_backend 'ExLlama V2'

[BlairSadewitz]

Aah, thanks! It is obvious in retrospect, I'm amused I didn't think of that.

[BlairSadewitz]

oh, BTW, FWIW, I got so annoyed at the fact that ./install_requirements.sh had to run the thing twice because of that bug I was like "Ok, that's IT. NEVER AGAIN. There MUST be a way to do this." And so I thought about how I created micromamba environments and changed it to do it this way, and it works. As to WHY, well, that is way above my pay grade ($0).

MAMBA_ROOT_PREFIX="${PWD}/runtime" if [[ $1 = "cuda" || $1 = "CUDA" ]]; then wget -qO- https://micromamba.snakepit.net/api/micromamba/linux-64/latest | tar -xvj bin/micromamba bin/micromamba create -f environments/huggingface.yml -r ${MAMBA_ROOT_PREFIX} -p ${MAMBA_ROOT_PREFIX}/envs/koboldai -y