DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

[WARNING] [cpu_adam.py:84:__init__] FP16 params for CPUAdam may not work on AMD CPUs

Open ninghongbo123 opened this issue 1 year ago • 2 comments

Is deepspeed compatible with AMD CPU ? When i import DeepSpeedCPUAdam optimizer on AMD CPU, I got following warning: [WARNING] [cpu_adam.py:84:init] FP16 params for CPUAdam may not work on AMD CPUs Can i just ingore the warning? Or it is imcompatible with AMD CPU ?

ninghongbo123 avatar Nov 17 '23 06:11 ninghongbo123

CC: @jithunnair-amd and @rraminen

loadams avatar Nov 17 '23 16:11 loadams

Noting here for context: this comes from PR https://github.com/microsoft/DeepSpeed/pull/2047

Some relevant discussion on why this was added: https://github.com/microsoft/DeepSpeed/pull/2047#discussion_r1401157111

It looks like some CPU vectorized instructions are used in https://github.com/microsoft/DeepSpeed/blob/master/csrc/includes/simd.h#L19

AVX256 and AVX512 are supported in the new AMD Ryzen CPUs. However, we need to check if all the vectorized instructions used in that file are supported on AMD CPUs.

jithunnair-amd avatar Apr 11 '24 19:04 jithunnair-amd