RuntimeError: The pointer[top_cell_] is null.

[Lyn0222]

问题描述 在调用mindnlp.peft微调的时候发生报错

Traceback (most recent call last):
  File "/tmp/code/biogpt/finetune.py", line 130, in <module>
    trainer.train()
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindnlp/engine/trainer/base.py", line 781, in train
    return inner_training_loop(
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindnlp/engine/trainer/base.py", line 1133, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs)
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindnlp/engine/trainer/base.py", line 1424, in training_step
    loss = self.grad_fn(inputs)
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindnlp/core/autograd/function.py", line 35, in value_and_grad_f
    values = fn_(*args, **kwargs)
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindnlp/engine/trainer/base.py", line 1416, in forward
    return self.compute_loss(model, inputs)
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindnlp/engine/trainer/base.py", line 1438, in compute_loss
    outputs = model(**inputs)
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindnlp/core/nn/modules/module.py", line 391, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindnlp/core/nn/modules/module.py", line 405, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindnlp/peft/peft_model.py", line 646, in forward
    past_key_values = self.get_prompt(batch_size)
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindnlp/peft/peft_model.py", line 319, in get_prompt
    prompt_tokens = prompt_tokens[:, : peft_config.num_virtual_tokens]
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindspore/common/tensor.py", line 473, in __getitem__
    out = tensor_operator_registry.get('__getitem__')(self, index)
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindspore/ops/composite/multitype_ops/_compile_utils.py", line 201, in _tensor_getitem
    new_index, tensor_update_types, tensor_update_args = getitem_tensor_index_info(
  File "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindspore/ops/operations/_inner_ops.py", line 2356, in __call__
    return Tensor_.getitem_index_info(data, index, self.is_ascend)
RuntimeError: The pointer[top_cell_] is null.

- Framework Unexpected Exception Raised:
----------------------------------------------------
This exception is caused by framework's unexpected error. Please create an issue at https://gitee.com/mindspore/mindspore/issues to get help.

----------------------------------------------------
- C++ Call Stack: (For framework developers)
----------------------------------------------------
mindspore/ccsrc/pipeline/pynative/grad/grad.h:75 top_cell

• Hardware Environment(Ascend/GPU/CPU) / 硬件环境:

Please delete the backend not involved / 请删除不涉及的后端: ASCEND910

• Software Environment / 软件环境 (Mandatory / 必填): -- MindSpore version :2.3.1 -- Python version :3.9.18 -- OS platform and distribution :Linux