Fabricio Bronzati

[FEATURE REQUEST] Multi node training support

1

### Feature Request Ability to train LLMs like Llama 2 70B and Falcon 180B on multi node configuration using Slurm or Kubernetes ### Motivation with maximum of 8x H100 GPUs...

Failing to build with triton 23.04

2

### Description ```shell System H100 and L40, driver 530.30.02 and cuda 12.1 Build is failing with following error, no mather the branch used (main, v1.4, fix/multi_instance, etc..) [ 54%] Built...