SangBin Cho

test result looks much more promising after the logging PR. Btw we should also change the logging doc. Maybe we can do it as a follow up (or within this...

cc @simon-mo

thanks for the quick review @Yard1 !

Looks like this actually fails tests for some reasons. will investigate it

seems not trivial. we are just going to add a separate test

hmm interestingly, when I use short prompt + word matching, it passes, but not with long prompt (not sure if it is sliding window is not working properly or long...

Intsead of long prompt, we are using short prompt now

Hmm wonder if there's a bug. Although the test uses bfloat16, I am seeing this error; ``` E RuntimeError: expected scalar type BFloat16 but found Half   ``` But I couldn't...

@simon-mo okay, I think there was a bug in our rotary embeding. I believe it is fixed.

ETA tmrw!