zfnice

In the demo you provided, nunchaku is used to load the model, but torch is still used for inference. How to use nunchaku custom operators for inference and acceleration?

使用原始脚本多卡推理以及单卡teacache时候没有问题，多卡就会报错。查到原因可能是权重加载至加载到了rank0上，请问怎么解决呢？ [rank1]: RuntimeError: setStorage: sizes [256, 5120], strides [1, 256], storage offset 47938560, and itemsize 4 requiring a storage size of 196997120 are out of bounds for storage of size...