Serge Rogatch

icon indicating a website link https://algoextreme.com/ icon indicating an email [email protected]

icon company Algorithmization Sp. z o.o. icon location Poznan, Poland icon location A C++ & Python programmer with primary skills in CUDA, Large Language Models, Cryptominers and Algorithms & Data Structures. MSc in AI from Amsterdam University

Results 5 comments of Serge Rogatch

Wrong parsing of relative URLs

Thanks, @lganzzzo , that works.

device_map='auto' doesn't use MPS backend on Apple M2

> MPS devices are indeed not supported with `device_map="auto"` yet. As a workaround you should just move your model to that device manually. How to move the model to that...

Multi-GPU bug?

> I use the following command to run normally on multi-gpu `CUDA_VISIBLE_DEVICES=0,1,2,3` accelerate launch finetune.py` But actually only gpu 0 and 1 are used. I don't know why Try to...

Support multi-GPU training

I have batch size 64, history length 1440, lookahead 480, and 2 million points in the time series, each consisting of 4 values. A single GPU is saturated 97-100% currently,...

Support multi-GPU training

Yes, we need to add `DistributedDataParallel` object, multi-processing launch, get the local rank of each process, and use it as the device parameter in `model.to()`. I planned to add this...