Infi_zc

Fix gradient and var name mismatch when num_groups > 0

4

## Checklist before submitting - [x ] Did you read the [contributor guide](https://github.com/horovod/horovod/blob/master/CONTRIBUTING.md)? - [x] Did you update the docs? - [x] Did you write any tests to validate this...

**Environment:** 1. Framework: (TensorFlow, Keras, PyTorch, MXNet) Tensorflow 2. Framework version: v1.15.4 3. Horovod version: v0.21.1 4. MPI version: 4.1.0 5. CUDA version: - 6. NCCL version: - 7. Python...

[QUESTION] Why should CUDA_DEVICE_MAX_CONNECTIONS=1 should be set when using seq_parallel or async comm?

5

I have a question, why is it necessary to set CUDA_DEVICE_MAX_CONNECTIONS=1 after enabling seq_parallel? This note is written in the bwd_compute function, and it says it is to launch communication...