How to improve the result? (already add beam search)

[trx14]

I want to improve transformer model. I think tensor2tensor is too big to change, so I choose this code. First, I have to reproduce the previous result. I add beam search by myself(before adding the beam search, the Bleu result was 28.34) and train model on iwslt14 de-en database(many papers use this database, not the iwslt16 de-en). I set all hyper parameter same as the base model in original paper. But I only get 30.34 Bleu, however many papers said they could get 32.86. Did some one know how to improve the result? Or did some other public transformer projects easy to change?

[yaoyiran]

Hi, I am also trying to add the beam search function. Would you mind sharing how you implemented the beam search part?

[ZhichaoOuyang]

Hello! Would you mind sharing how you implemented the beam search part? I want to learn it.

[ZhichaoOuyang]

I want to improve transformer model. I think tensor2tensor is too big to change, so I choose this code. First, I have to reproduce the previous result. I add beam search by myself(before adding the beam search, the Bleu result was 28.34) and train model on iwslt14 de-en database(many papers use this database, not the iwslt16 de-en). I set all hyper parameter same as the base model in original paper. But I only get 30.34 Bleu, however many papers said they could get 32.86. Did some one know how to improve the result? Or did some other public transformer projects easy to change?

Hello! Would you mind sharing how you implemented the beam search part?