transformer-models icon indicating copy to clipboard operation
transformer-models copied to clipboard

GPT-2 doesn't include dropout layers

Open bwdGitHub opened this issue 3 years ago • 0 comments

We would like to use these issues to gauge user interest.

The GPT-2 implementation does not include dropout layers. This would be useful for further pre-training and fine-tuning workflows to prevent overfitting.

bwdGitHub avatar Dec 10 '21 16:12 bwdGitHub