Andrei Panferov comments

Repositories
Issues
Comments

Results 41 comments of


                                            Andrei Panferov

tensor_parallel method distributed=True

With `distributed=True` you pass the device associated with current worker. For example, if you have 4 _GPUs_, you launch with `--nproc_per_node=4`, extract local worker rank with `LOCAL_RANK = int(os.environ['LOCAL_RANK'])` and...