memnoh

Multi-node training seems to be even slower than single node

1

Hi all, I've got a 2 node setup with 1 1080 Titan each. I've successfully managed to configure MPI with NCCL and run marian on both nodes (with sync-sgd). For...