Yang Liu

Training problem

1

@ConnorJL Thanks for the great work. Unfortunately, I found out my training using OpenWebTextCorpus is too slow even for 117M model. The cross entropy loss function decreases rapidly before 10k...