BELLE icon indicating copy to clipboard operation
BELLE copied to clipboard

feature-request: publish half-precision models

Open peakji opened this issue 1 year ago • 1 comments

The original bigscience/bloomz-7b1-mt model was released in half-precision (torch.HalfStorage), so its weight file is only 14.1 GB in size. I noticed that the current Belle weights are released intorch.FloatStorage, so the file size is twice the size of the foundation model.

Is it possible to publish a variant of Belle in half-precision? It would make it easier for everyone to try it out.

peakji avatar Mar 20 '23 03:03 peakji