Setup CI to check on 2 GPUs

[vfdev-5]

As CircleCI removed resource class with 2 GPUs we can not run tests on 2 GPUs with Circle CI:

• https://app.circleci.com/pipelines/github/pytorch/ignite/2461/workflows/5686c0d7-6dcc-4693-926a-3d91b1e92119/jobs/7650

Resource class "gpu.medium" is not a valid resource class. The default resource class will be used.

Available resource classes are only 1 GPU: https://circleci.com/docs/2.0/configuration-reference/#gpu-executor-linux

There is a feature request on below link (but IMO it is a regression):

• https://circleci.canny.io/cloud-feature-requests/p/multi-gpu-resources

I submitted a request:

• https://support.circleci.com/hc/en-us/requests/104517

We can think of using AWS as PyTorch does or setup a custom runner on our own multi-gpu servers.

cc @sdesrozis @trsvchn @fco-dv

---

• [x] Set up GHA self-hosted runner and ported unit tests on 2 GPUs and check the training on CIFAR10
• [ ] Port HVD tests on 2 GPUs from config.yml