safetensors icon indicating copy to clipboard operation
safetensors copied to clipboard

converting weight of more than 50G‘s .bin to .safetensors failed

Open WongGawa opened this issue 6 months ago • 0 comments

System Info

I am using https://huggingface.co/spaces/safetensors/convert,converting google/mt5-xxl 's .bin file to .safetensors file,but it occurs error because .bin is more than 50G。 I am wondering how can this problem be solved? Thank you。

Information

  • [X] The official example scripts
  • [ ] My own modified scripts

Reproduction

1、https://huggingface.co/spaces/safetensors/convert 2、google/mt5-xxl 3、submit

Expected behavior

No error and get one pr。take google/mt5-xl for example。 https://huggingface.co/google/mt5-xl/discussions/2

WongGawa avatar Aug 21 '24 02:08 WongGawa