Is it possible to support microsoft/bitnet-b1.58-2B-4T ?

[hbj52152]

Reminder

• [x] I have read the above rules and searched the existing issues.

System Info

i have simple check the latest version, and i have check the issue with word 'bitnet' but no relating informaiton, and also in model list.

Sorry if i missed any information.

Reproduction

model info

A new and small model,

microsoft/bitnet-b1.58-2B-4T

https://huggingface.co/microsoft/bitnet-b1.58-2B-4T

detials found

i am not sure whether it is suitable to train with LLAMA-Factory, i notice that there is informaiton from huggingface: microsoft/bitnet-b1.58-2B-4T-bf16: Contains the master weights in BF16 format. Use this only for training or fine-tuning purposes.

other check

no more informaiton about trainning from both huggingface and https://github.com/microsoft/BitNet.

Others

No response

[yuimo]

+1