[DOUBT] Unofficial Quantized Model in HuggingFace Repo Safe and Perfect to use ?

[Greatz08]

There are no Official Quantized Model So Will Team Support Quantized Version Official or Not ? As you can see Following are the only unofficial quantized models available so i thought to ask regarding that like is it safe to use ? Will it provide similar performance ? Has anyone from team or community member tested those ones out ?

[tmlx1990]

Hello, the official AI model can be downloaded here: https://github.com/CodePhiliaX/Chat2DB-GLM?tab=readme-ov-file

[Greatz08]

@tmlx1990 yeah i did saw this too but i want quantized (compressed) version of model as most of them wont be able to run this model locally as most have 8-10GB vram only so i and hopefully many others will be searching for quantized version of original model so if possible please create different quantized models which we can also run on our local system easily. Q4,Q5 K_L and K_M quantized model variants will be best for run under 8-10 GB VRAM without too much compromise on quality of model .