dialog-nlu icon indicating copy to clipboard operation
dialog-nlu copied to clipboard

Way lower accuracy than example notebooks

Open J0ekr opened this issue 2 years ago • 1 comments

Hey, Thanks for the great repo! I am currently trying to train a NLU and tested out your using_bert_crf_nlu example notebook with your provided data. I have tested both Bert and Albert embeddings. However, I always only get 14% accuracy (so random classification), while you have ~98% accuracy. I am currently using python 3.11 - the only major compatibility issue I had was with sentencepiece (dependency from transformers), where I now use the newest version instead of your specified version range. I can't imagine this being the case for the model not working anymore. I will continue investigating this, but wanted to ask whether you or someone else already had this issue and has ideas!

Thanks!

J0ekr avatar Jul 12 '23 13:07 J0ekr