Logan Adams
Logan Adams
Thanks @weiqisun - this PR was merged so I'm closing this issue for now.
@xuedinge233 - can you please address the formatting test failure?
Hi @csxrzhang - can you explain more on your issue? Are you still looking for code or would you like to make a PR if you have a code improvement/suggestion?
Hi @csxrzhang - closing this issue as stale for now.
Hi @JKYtydt - the tutorial on inference for DeepSpeed [here](https://www.deepspeed.ai/tutorials/inference-tutorial/#launching) includes directions on how to invoke DeepSpeed such that it can support multi-node inference.
Hi @zhiwentian - you will need to install numactl, depending on your linux distro it might be `apt-get install numactl` - could you try that?
Hi @zhiwentian - we would need to have more of your repro script, but it looks like fp16 isn't supported, can you share what device you are running on?
@zhiwentian - are you still hitting this issue? It looks like you're hitting this check, whichi s weird that fp16 wouldn't be supported on your accelerator, could you check that?...
Hi @zhiwentian - closing this for now, if you are still hitting this or someone hits this in the future please comment and we can debug more/re-open.
Thanks @hijkzzz and @thomZ1 for narrowing down the problematic commit. Tagging @tjruwase and @wenbinc-Bin for the original PR to take a look as well.