added the option to use a custom model or embedding instead of the hardcoded ones

[Allaye]

• added cli option to specify embedding model, LLAMA, and AutoGTPQ model
• annotate the functions with expected argument types

[Tchekda]

The embedding should be configurable in both ingest.py and run_localGPT.py as in #139

[PromtEngineer]

@Allaye When we are adding the model choice to CLI, we have the default model_basename set to WizardLM-7B-uncensored-GPTQ-4bit-128g.compat.no-act-order.safetensors. The unquantized models do not have the model_basename and I think it will cause issues when we try to load the model. Probably its better to add a check on the model type when we are calling the load_model function. What do you think?

[Allaye]

I will look into it..

On Sun, Jun 11, 2023, 11:48 PM PromptEngineer @.***> wrote:

@Allaye https://github.com/Allaye When we are adding the model choice to CLI, we have the default model_basename set to WizardLM-7B-uncensored-GPTQ-4bit-128g.compat.no-act-order.safetensors. The unquantized models do not have the model_basename and I think it will cause issues when we try to load the model. Probably its better to add a check on the model type when we are calling the load_model function. What do you think?

— Reply to this email directly, view it on GitHub https://github.com/PromtEngineer/localGPT/pull/138#issuecomment-1586364801, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEEPRYRZYCOVDK3N2PEUZ5TXKZDLZANCNFSM6AAAAAAZCQVHGE . You are receiving this because you were mentioned.Message ID: @.***>

[Allaye]

@PromtEngineer i think that check is been done in the load_model function already.

[PromtEngineer]

@Allaye based on the if condition in the load_model function, irrespective of whether the model is quantized or not, since we will be providing a default model_basename, it will always use the AutoGPTQForCausalLM for loading the model. That's what's seems to be happening. Can you double-check?

[Allaye]

@Allaye based on the if condition in the load_model function, irrespective of whether the model is quantized or not, since we will be providing a default model_basename, it will always use the AutoGPTQForCausalLM for loading the model. That's what's seems to be happening. Can you double-check?

oh okay

[Allaye]

@PromtEngineer currently if a non_quantized model is used even with the default model_basename one of the elif or else will used.

[Allaye]

@PromtEngineer can you take a look at this now??

[PromtEngineer]

@Allaye thanks for the update, I will have a detailed look at it later today and will merge it if I dont' see any further changes that need to be made. Thanks for the help.

[PromtEngineer]

@Allaye, sorry was out. The PR is failing the pre-commit. can you please look at it? The update looks good to me and I will merge it once this is resolved. Thanks,