gpt-fast icon indicating copy to clipboard operation
gpt-fast copied to clipboard

Published 20 hours ago • verified publisher icon pytorch-labs
→

Metadata

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

Results 132 gpt-fast issues
Sort by recently updated
recently updated
newest added

Error with stories15M and stories110M

8 comment

When I use stories15M and stories110M I got an error. ``` File "D:\_LLM_project\Development\gpt-fast\generate.py", line 114, in speculative_decode torch.cat([cur_token.view(1), draft_tokens]).view(1, -1), RuntimeError: Tensors must have same number of dimensions: got 1...

deafTim avatar deafTim

Request for Smaller Model Options (~1B Parameters)

Currently, I can only use small models. Is there an option to find an appropriate model around the 1B parameter size?

deafTim avatar deafTim

← Metadata

4.3k
Stars
447
Forks
49
Watchers

Owner

verified publisher icon pytorch-labs

Metadata

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

Back