Mandy Li

With commit ID 2d8b0cb9b2e70281bf9dce438ff17ffa5e59075c, if use batch size other than 1 to do inference, an exception will be thrown. This is the exception I got with batch size 2 for...

Fix RoPE data type issue for gpt_neox and stablelm (#177)

2

Internal implementation for RoPE changed so that if one of parameters data type is FP32, the op will be performed in FP32 data type. To force the op to bf16,...

Fix gpt_neox, stablelm inference regression caused by RoPE dtype

3

As title