OSError: (External) CUDA error(719), unspecified launch failure

[JohnsenJiang]

问题确认 Search before asking

• [X] 我已经搜索过问题，但是没有找到解答。I have searched the question and found no related answer.

请提出你的问题 Please ask your question

python tools/train.py -c jswork\configs\yolov8\yolov8_s_500e_coco.yml 报如下错误信息： [06/24 13:09:51] ppdet.data.source.coco INFO: Load [132 samples valid, 0 samples invalid] in file E:/Data/QJU006A_LAB/coco\annotations/instance_train.json. W0624 13:09:51.744545 29832 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 8.9, Driver API Version: 12.5, Runtime API Version: 11.8 W0624 13:09:51.754547 29832 gpu_resources.cc:164] device: 0, cuDNN Version: 8.6. Error: ../paddle/phi/kernels/funcs/gather.cu.h:67 Assertion index_value >= 0 && index_value < input_dims[j] failed. The index is out of bounds, please check whether the dimensions of index and input meet the requirements. It should be less than [2] and greater than or equal to 0, but received [0] Error: ../paddle/phi/kernels/funcs/gather.cu.h:67 Assertion index_value >= 0 && index_value < input_dims[j] failed. The index is out of bounds, please check whether the dimensions of index and input meet the requirements. It should be less than [2] and greater than or equal to 0, but received [0] Error: ../paddle/phi/kernels/funcs/gather.cu.h:67 Assertion index_value >= 0 && index_value < input_dims[j] failed. The index is out of bounds, please check whether the dimensions of index and input meet the requirements. It should be less than [2] and greater than or equal to 0, but received [0] Error: ../paddle/phi/kernels/funcs/gather.cu.h:67 Assertion index_value >= 0 && index_value < input_dims[j] failed. The index is out of bounds, please check whether the dimensions of index and input meet the requirements. It should be less than [2] and greater than or equal to 0, but received [0] Error: ../paddle/phi/kernels/funcs/gather.cu.h:67 Assertion index_value >= 0 && index_value < input_dims[j] failed. The index is out of bounds, please check whether the dimensions of index and input meet the requirements. It should be less than [2] and greater than or equal to 0, but received [0] File "D:\Project\Paddle\PaddleYOLO-develop\ppdet\modeling\heads\yolov8_head.py", line 276, in get_loss self.assigner( File "E:\ProgramData\Anaconda3\envs\pd2.6.0\lib\site-packages\paddle\nn\layer\layers.py", line 1429, in call return self.forward(*inputs, **kwargs) File "E:\ProgramData\Anaconda3\envs\pd2.6.0\lib\site-packages\decorator.py", line 232, in fun return caller(func, *(extras + args), **kw) File "E:\ProgramData\Anaconda3\envs\pd2.6.0\lib\site-packages\paddle\base\dygraph\base.py", line 352, in _decorate_function return func(*args, **kwargs) File "D:\Project\Paddle\PaddleYOLO-develop\ppdet\modeling\assigners\task_aligned_assigner.py", line 148, in forward is_in_topk = gather_topk_anchors( File "D:\Project\Paddle\PaddleYOLO-develop\ppdet\modeling\assigners\utils.py", line 105, in gather_topk_anchors return is_in_topk * topk_mask OSError: (External) CUDA error(719), unspecified launch failure. [Hint: 'cudaErrorLaunchFailure'. An exception occurred on the device while executing a kernel. Common causes include dereferencing an invalid device pointerand accessing out of bounds shared memory. Less common cases can be system specific - more information about these cases canbe found in the system specific user guide. This leaves the process in an inconsistent state and any further CUDA work willreturn the same error. To continue using CUDA, the process must be terminated and relaunched.] (at ..\paddle\phi\backends\gpu\cuda\cuda_info.cc:272)

平台：win10 显卡：RTX4060 laptop Cuda：11.8 Cudnn：8.6.0 Paddle：2.6.0