Amir Zait comments

Repositories
Issues
Comments

Results 1 comments of


                                            Amir Zait

Implement Together Computer's Red Pajama 3B Base/Chat model

The problem is that stableLM uses the gptneoX with use_parallel_residual=True (so that each block is x + mlp(x) + attn(x)) and RedPajama uses use_parallel_residual=False. I implemented it in https://github.com/amirza1/ggml and...