Blake

@RezaYazdaniAminabadi Thanks for looking into this. Whatever the issue is, I have seen many reports of issues so there seems to be a new requirement to successfully install or an...

Tried to compile with version 0.8.2 and it still does not work.

Read into this more. I guess I don't have to compile it this way and can just use JIT compilation. Still an odd issue.

> @mallorbc - Do things work fine with JIT compilation? You're just not able to build the ops? Especially since you can build the ops on 0.8.0 but not 0.8.1...

Inference seems to work as well at least for fp16. Int8 does not work, but that has been an issue for a while. When I try to use int8 I...

@loadams Thanks for looking into this issue, I am guessing you built single ops at a time? Is there a downside to using JIT? Perhaps it is slower than precompiling...

@loadams makes sense why that may cause issues. I will try that PR of the fix when I get a chance and let you know what I find. Thanks!

I am also having issues: I am using Ubuntu 20.04. Compiling version 0.8.1 with my docker image as you can see [here](https://github.com/mallorbc/Finetune_GPTNEO_GPTJ6B/blob/main/Dockerfile) also leads to issues. This works: ``` DS_BUILD_OPS=1...

@tokestermw is Beam Search still broken with DeepSpeed? You asked about contrastive search, and due to the issue here https://github.com/huggingface/transformers/issues/21151 It seems that contrastive search is broken

@tokestermw Thanks so much for sharing your insights! I assume to get these results you did something like a string compare for results generated with and without DeepSpeed?