AutoAWQ icon indicating copy to clipboard operation
AutoAWQ copied to clipboard

Published 20 hours ago • verified publisher icon casper-hansen

How to use multiple GPU nodes during quantization

Open Juniper1021 opened this issue 11 months ago • 2 comments

When I am converting my Qwen2VL-72B model, I want to use multiple GPU nodes to utilize more data. How to achieve this

Juniper1021 avatar Dec 12 '24 14:12 Juniper1021

Hi @ghntd, at the moment, data parallelism is not implemented. I welcome any help on implementing this that demonstrates a speedup.

casper-hansen avatar Dec 12 '24 15:12 casper-hansen

@casper-hansen any update on this?

radna0 avatar Mar 02 '25 00:03 radna0