刘迅承 issues

Repositories
Issues
Comments

Results 1 issues of


                                            刘迅承

推理时报错RuntimeError: CUDA error: device-side assert triggered

使用的模型是003的plugin int4模型，显卡是3090，报错时显存显示占用了12.5G。具体报错信息如下： /opt/conda/conda-bld/pytorch_1670525552843/work/aten/src/ATen/native/cuda/IndexKernel.cu:92: operator(): block: [272,0,0], thread: [127,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed. Exception in thread Thread-7 (generate): Traceback (most...