RichardWooSJTU

Results 3 comments of RichardWooSJTU

paddle inference: can the input already be on GPU, is there any interface provided

Sorry, due to many considerations, we don't have such an interface for the time being. Please describe the usage scenario in detail for us to discuss. Thank you!

paddle inference: can the input already be on GPU, is there any interface provided

Thank you for your reply, we will discuss seriously.

[Question]: paddlenlp_ops安装报错PaddleNLP/csrc/generation/quant_int8.cu(57): error: no suitable user-defined conversion from "__nv_bfloat16" to "__half" exists

从报错中发现您的环境应该是V100机器，cuda版本应该不会是11.2（请通过nvcc -V确认而非nvidia-smi）.您的问题应该是由于cuda版本不支持bf16导致的，抱歉我们最近的修改没有对BF16进行完整测试，我们将很快对此进行修复