yanqiangmiffy

ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 1

4

ERRO: ``` Using .cache/torch_extensions/py310_cu117 as PyTorch extensions root... Emitting ninja build file .cache/torch_extensions/py310_cu117/utils/build.ninja... Building extension module utils... Allowing ninja to set a default number of workers... (overridable by setting the...

ptuning评估的时候报错：The expanded size of the tensor (140) must match the existing size (312) at non-singleton dimension 0. Target sizes: [140]. Tensor sizes: [312]

5

### Is there an existing issue for this? - [X] I have searched the existing issues ### Current Behavior ``` File "/ptuning/main.py", line 416, in main() File "/ptuning/main.py", line 354,...

ValueError: ChatGLMForConditionalGeneration does not support gradient checkpointing.

8

` File "finetune.py", line 128, in main model.gradient_checkpointing_enable()` 这个有问题大家有遇到吗

做了中文Belle的实验

3

https://github.com/yanqiangmiffy/InstructGLM 不过modeling_chatglm.py用的官方的，之前对比过实验效果，发现微调alpaca数据之后，官方的效果之后好一点 修改modeling_chatglm的结果：  官方modeling_chatglm结果： 

FastSpeech 模型下载链接失效

4

FastSpeech | https://pan.baidu.com/s/18WphipZTW-aNY82ISlU8Nw | tjru | 28M | 0.056 -- | -- | -- | -- | --

# PR type - [ ] Bug Fix - [x] New Feature - [ ] Document Updates - [ ]More Models or Datasets Support # PR information ``` dataset_info =...

分布式保存模型，需要model.module.state_dict()

2

https://zhuanlan.zhihu.com/p/447563272 保存模型： ``` # 保存模型 if dist.get_rank() == 0: validate() # 在测试集上验证模型性能 state_checkpoint = { 'state_dict': model.module.state_dict(), 'optimizer':optimizer.state_dict()} torch.save(state_checkpoint, model_name) ```