ucaslei

Results 2 comments of ucaslei

[Question] 单机单卡训练，报错，无法初始化梯度。

> > 我也遇到了同样的问题，在deepspeed issues中有找到相关说明[https://github.com/microsoft/DeepSpeed/issues/3234，ZeRO](https://github.com/microsoft/DeepSpeed/issues/3234%EF%BC%8CZeRO) stage 3支持zero.init，stage 1和2不支持，我把deepspeed.json中stage改成3解决了这个问题 > > ![image](https://private-user-images.githubusercontent.com/108161275/294867193-00667787-3e62-4d38-a3cb-58028ab740ea.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjI5MjUxMjQsIm5iZiI6MTcyMjkyNDgyNCwicGF0aCI6Ii8xMDgxNjEyNzUvMjk0ODY3MTkzLTAwNjY3Nzg3LTNlNjItNGQzOC1hM2NiLTU4MDI4YWI3NDBlYS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwODA2JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDgwNlQwNjEzNDRaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0wYzY5ZmVmZjExY2FhNGEyNDkyZTU5ZDAwMDY2ZDRiNzI1MjJmOTMwNzY3ODJjOGMwNWIwZjllOGQxZTJiOWQwJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.Rhvwc7t6R4x5Jq7dcZK8uVzgKRsBIYKvbXIljcrv5L0) 按你的方法修改后有新的报错，你有遇到吗 我也遇到这个问题了 有解决方法吗

Support for Multiple Datasets and Domain-Specific Loss Calculation in Trainer

+1