yet-another-retnet
yet-another-retnet copied to clipboard
fkodom
A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (https://arxiv.org/pdf/2307.08621.pdf)
Hi again (: I've found a small problem in the current implementation of the initiallization of the `RetNetDecoder` class. Specifically, to build a multi-layered model, this class uses `deepcopy` to...
I have committed some minor changes to a new branch and attempted to push these, but I am getting access denied.
I tried to do inference but got the above error. I'm using python3.10 on Ubuntu 22.04. i9-13900K + 4090 I fixed this by changing return cls(**checkpoint_dict) to return cls(**vars(checkpoint_dict)) ```...
