WenjingBao

Results 9 comments of WenjingBao

在看文档时说训练sft模型时 需要将该 token 指定为<eom>，但是在哪里改呢？

我没搞错的话应该是finetune_moss.py的178行 `tokenizer.eos_token_id = 106068 # The eos_token_id of base model is 106028. We need map the eos token to (its token id is 106068)`

请教 RuntimeError: `<class 'models.quantization.QuantLinear'>' was not properly set up for sharding by zero.Init(). A subclass of torch.nn.Module must be defined before zero.Init() where an instance of the class is created.

遇到了相同的问题，期待解答

请教 RuntimeError: `<class 'models.quantization.QuantLinear'>' was not properly set up for sharding by zero.Init(). A subclass of torch.nn.Module must be defined before zero.Init() where an instance of the class is created.

嗯，我这里解决了，是把run.sh里面--model_name_or_path这行改成本地地址的时候没在前面加 `./` 加上了就好了。。。

请教 RuntimeError: `<class 'models.quantization.QuantLinear'>' was not properly set up for sharding by zero.Init(). A subclass of torch.nn.Module must be defined before zero.Init() where an instance of the class is created.

> > 嗯，我这里解决了，是把run.sh里面--model_name_or_path这行改成本地地址的时候没在前面加 > > `./` > > 加上了就好了。。。 > > 哇，所以可以单卡训练量化模型是吗？请问一下你训练的是哪个量化模型呢，用的卡是什么？ 应该是，我还在解决后续出现的别的bug...

请教 RuntimeError: `<class 'models.quantization.QuantLinear'>' was not properly set up for sharding by zero.Init(). A subclass of torch.nn.Module must be defined before zero.Init() where an instance of the class is created.

刚发现之前做的并不能解决问题，只是清掉了cache导致新的bug更早出现了（捂脸） 不过问题好像是出在 `./models/quantization.py` 里面295行的QuantLinear这个class里 以及deepspeed的repo里面有个类似问题的issue： [DeepSpeed/issues/2812](https://github.com/microsoft/DeepSpeed/issues/2812) 但里面的解决方法好像比较复杂，还得继续看看

请教 RuntimeError: `<class 'models.quantization.QuantLinear'>' was not properly set up for sharding by zero.Init(). A subclass of torch.nn.Module must be defined before zero.Init() where an instance of the class is created.

这次是真的解决了，我拿conda给finetune单独建了一个env，安装了下列package `pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu117 pip install pandas accelerate==0.17.1 numpy==1.24.2 regex==2022.10.31 tqdm==4.64.1 transformers==4.25.1 deepspeed tensorboard conda install jupyterlab=3.5.3 -c conda-forge` 然后先 `accelerate test --config_file ./configs/sft.yaml` 生成cache 接着手动把...

请教 RuntimeError: `<class 'models.quantization.QuantLinear'>' was not properly set up for sharding by zero.Init(). A subclass of torch.nn.Module must be defined before zero.Init() where an instance of the class is created.

> 这次是真的解决了，我拿conda给finetune单独建了一个env，安装了下列package > > `pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu117 > > pip install pandas accelerate==0.17.1 numpy==1.24.2 regex==2022.10.31 tqdm==4.64.1 transformers==4.25.1 deepspeed tensorboard > > conda install jupyterlab=3.5.3 -c conda-forge`...

请教 RuntimeError: `<class 'models.quantization.QuantLinear'>' was not properly set up for sharding by zero.Init(). A subclass of torch.nn.Module must be defined before zero.Init() where an instance of the class is created.

找到一个多卡/单卡训int8的示例code https://github.com/yangzhipeng1108/moss-finetune-and-moss-finetune-int8

请问有支持ARM架构VPS的计划吗？

> ProxySU能否支持arm架构的vps，其实与所安装的代理是否支持有关。naive是支持的，但是ProxySU所选用的版本不支持，这个以后会加以改进。你可以先尝试用别的类型代理试试。xray。v2ray应该都是支持的。 期待未来更新