Jeff Rasley

Closing, the original issue is resolved and new issue is moved to #2227

@tomeras91 is this still an issue? if so please re-open.

Hi @gtvforever, this is a bit not obvious on the HF side. Here is how we've accomplished this (with or without deepspeed): ```python from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline model_name...

@gtvforever please re-open if this isn't resolved.

@hamzafar, can you share the results of this? For some reason torch might not be detecting your CUDA_HOME path. `python -c "import torch.utils.cpp_extension; print(torch.utils.cpp_extension.CUDA_HOME)"`

Interesting, do you have `nvcc` installed somewhere on your machine? This should come with the cuda toolkit (e.g, https://developer.nvidia.com/cuda-11.3.0-download-archive). If you have nvcc, can you share the version? e.g., `nvcc...

I suspect that is the source of the issue here, DeepSpeed requires `nvcc` to be installed to compile our c++/cuda extensions. I just created a PR (https://github.com/microsoft/DeepSpeed/pull/2519) that adds a...

Interesting, glad to see `nvcc` is coming up okay. This error at install makes me think CUDA_HOME is still returning None from torch though. Does `python -c "import torch.utils.cpp_extension; print(torch.utils.cpp_extension.CUDA_HOME)"`...

What version of torch do you have installed here? can you show me the exact `torch.__version__`? I just want to make sure torch cuda is aligned with cuda 10.0 you...

@SuhitK, please re-open if you are still having issues.