nlp-tutorial icon indicating copy to clipboard operation
nlp-tutorial copied to clipboard

Published 20 hours ago • verified publisher icon graykode

Problem with BERT batch generation

Open aqibsaeed opened this issue 4 years ago • 1 comments

There is a problem with padding on line 73-75 . What if the sentence length is larger than maxlen? Then we end up with sequences of varying length and line 214 throws an error.

aqibsaeed avatar Apr 15 '20 10:04 aqibsaeed

You cannot have sentences having length greater than maxlen. In cases where the length of a sentence is lesser than maxlen, the code applies padding.

Soothysay avatar Jan 10 '23 02:01 Soothysay