PromptEngineer

@Allaye, sorry was out. The PR is failing the pre-commit. can you please look at it? The update looks good to me and I will merge it once this is...

@teleprint-me I agree we need to do a much better job at documentation and provide guides to the users for different platforms. I agree, we can provide a guide for...

@teleprint-me I agree with your points. These are the same things I would like to add. For the time being, I would like to merge #180 when you are able...

@endolith I will have a look at the code and see what is causing this.

@ChristianWeyer this seems to be a bug, thanks for highlighting it. I am not sure if auto_gptq supports M1/M2. Will need to test that.

@ChristianWeyer I finally got a M2 and just tested it, that is the case. Need to figure out if there is another way.

Just pushed a fix for it. Let me know if there is still the same issue. > BTW @PromtEngineer: the current code checks for CUDA explicitly for full models, which...

@adjiap In your case the bottleneck is the RAM you have. You probably want to set the RAM [here ](https://github.com/PromtEngineer/localGPT/blob/925d63c66790d3787e58bbfd966797e03c2533f5/run_localGPT.py#L83C3-L83C3).

@VTaPo which OS are you running this on? For some reasons I am seeing this on MacOS but not on Linux.

@imjwang this is really helpful. I will be merging a major code change over next couple of days, can you please look into this afterwards? Thanks