PyTorch-LIT

PyTorch-LIT copied to clipboard

gpt-j generation speed very low

3

The output of gpt-j is very slow, for a 200 output token generation it takes about 20 minutes, for 2048 it takes more than an hour, this significantly limits any...

ghost

Weights file format is changed, function partial_loader fails

4

Hi, thanks for your effort for making it easy to load and do inference from large models. I tried your code on a gpt-j model with different model file format,...

ghost