Sunghwan Shim

Same kind of problem occurs when I run [`generate_text.sh` here](https://github.com/microsoft/Megatron-DeepSpeed/blob/main/examples/generate_text.sh). I've also tried the same thing with `nvcr.io/nvidia/pytorch:20.12-py3` docker image, but same error occured. **error log** ``` . . ....

I managed to create `InferenceEngine` by adding some configs, but other problem occurs when running forward pass of it. Following is the revised `pretrain_gpt.py`: ```python from megatron.training import initialize_megatron, get_model,...

@awan-10 Thanks for the comment. Unfortunately, I've already tried the example you shared and found it didn't work (https://github.com/microsoft/DeepSpeed/issues/2030#issuecomment-1193909540).

I've also tried this on a machine with v100 32G * 8, but failed with almost same error. Does the script only run on A100?

해당 library가 std에 의존합니다. 다른 좋은 library를 찾지 못해서 일단 보류하겠습니다.