What if turning off the highway connection?

[byzhang]

Just curious, how much would it impact the accuracy and speed (w/ roughly same amount of parameters)? Have you done some experiments on it?

Thanks!

[taolei87]

@byzhang

hi, I haven't done very thorough experiments on this. I noticed highway helps quite a bit in language modeling task in our earlier ICML work. So I stick with it.

It is in our TO-DO now -- comparing highway, residual (identity) and the version w/o skip.

[byzhang]

thanks. I'd love to see your new exps :)

Thanks, -B

On Wed, Sep 13, 2017 at 8:08 AM, taolei87 [email protected] wrote:

@byzhang https://github.com/byzhang

hi, I haven't done very thorough experiments on this. I noticed highway helps quite a bit in language modeling task in our earlier ICML work. So I stick with it.

It is in our TO-DO now -- comparing highway, residual (identity) and the version w/o skip.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/taolei87/sru/issues/9#issuecomment-329198250, or mute the thread https://github.com/notifications/unsubscribe-auth/AA1O1Xq-ZGnmiqpBHrZInlAlo5QNO7voks5sh-_TgaJpZM4PVhGB .