BitNet icon indicating copy to clipboard operation
BitNet copied to clipboard

changing the quantization to CPU specific

Open labKnowledge opened this issue 4 months ago • 0 comments

Improve model conversion reliability by defaulting to TL2 quantization

During testing, the I2_S quantization method frequently failed when converting the model to GGUF format. The conversion only succeeded when explicitly specifying CPU quantization TL2.

To enhance the reliability of the testing process, this PR proposes changing the default quantization method from I2_S to TL2. This change should make it easier for users to test the model without encountering conversion errors.

Action taken: 1.Updated the README to set TL2 as the default quantization method instead of I2_S on README

labKnowledge avatar Oct 19 '24 20:10 labKnowledge