How to train using main.py using multiple GPUs?

[alvations]

@yikangshen @shawntan Is there an easy way to train the model to replicate the experiments using main.py using multiple GPUs?

When using model = nn.DataParallel(model) before train(), the initialization goes into the LSTM stack and then the ONLSTM cell to return the weights but it throws an error.

We also tried doing the model = nn.DataParallel(model) after the hidden = model.init_hidden(args.batch_size) and it seems like the LinearDropConnect layer can't access the .weight tensors.

[yikangshen]

We didn't try to train the model with multiple GPUs. Maybe you need to rewrite the code for LinearDropConnect function

[BuaaAlban]

Another question, it seems no speed up using GPU compared with CPU, have you met the same problem? Both take 280-290 s each epoch

[Shiweiliuiiiiiii]

@yikangshen @shawntan Is there an easy way to train the model to replicate the experiments using main.py using multiple GPUs?

When using model = nn.DataParallel(model) before train(), the initialization goes into the LSTM stack and then the ONLSTM cell to return the weights but it throws an error.

We also tried doing the model = nn.DataParallel(model) after the hidden = model.init_hidden(args.batch_size) and it seems like the LinearDropConnect layer can't access the .weight tensors.

Hi, Just want to know have you figured this out? Best