RuntimeError: attn_softmax_inplace_forward_ not implemented on CPU

[ToHanwei]

Hi, I encountered this error while running inference.

modelling for Base185 INFO:/data1/hanwei/openfold/openfold/utils/script_utils.py:Successfully loaded JAX parameters at openfold/resources/params/params_model_1.npz... INFO:run_pretrained_openfold.py:Using precomputed alignments for Base185 at /data1/hanwei/openfold/20230314_base_modelling/outputs/alignments... INFO:/data1/hanwei/openfold/openfold/utils/script_utils.py:Running inference for Base185... Traceback (most recent call last): File "run_pretrained_openfold.py", line 391, in main(args) File "run_pretrained_openfold.py", line 254, in main out = run_model(model, processed_feature_dict, tag, args.output_dir) File "/data1/hanwei/openfold/openfold/utils/script_utils.py", line 159, in run_model out = model(batch) File "/data1/hanwei/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/data1/hanwei/openfold/openfold/model/model.py", line 515, in forward _recycle=(num_iters > 1) File "/data1/hanwei/openfold/openfold/model/model.py", line 373, in iteration _mask_trans=self.config._mask_trans, File "/data1/hanwei/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/data1/hanwei/openfold/openfold/model/evoformer.py", line 1006, in forward m, z = b(m, z) File "/data1/hanwei/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/data1/hanwei/openfold/openfold/model/evoformer.py", line 525, in forward self.ckpt if torch.is_grad_enabled() else False, File "/data1/hanwei/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/data1/hanwei/openfold/openfold/model/msa.py", line 273, in forward flash_mask=mask, File "/data1/hanwei/openfold/openfold/model/msa.py", line 125, in _chunk no_batch_dims=len(m.shape[:-2]) File "/data1/hanwei/openfold/openfold/utils/chunk_utils.py", line 299, in chunk_layer output_chunk = layer(**chunks) File "/data1/hanwei/openfold/openfold/model/msa.py", line 108, in fn flash_mask=flash_mask, File "/data1/hanwei/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in call_impl return forward_call(*input, **kwargs) File "/data1/hanwei/openfold/openfold/model/primitives.py", line 492, in forward o = attention_core(q, k, v, *((biases + [None] * 2)[:2])) File "/data1/hanwei/openfold/openfold/utils/kernel/attention_core.py", line 50, in forward attention_logits.shape[-1], RuntimeError: attn_softmax_inplace_forward not implemented on CPU

[Qiuicci]

+1

[egurapha]

+1

[jozhang97]

I had the same issue, likely did not set CUDA_HOME or other cuda environment variables correctly, so it installed cpu version.

You can check the first line output when running python setup.py install. https://github.com/aqlaboratory/openfold/blob/b5fa2ba316c2ed096fac05390076a2682819fb5f/setup.py#L42

[egurapha]

@jozhang97 I tried reinstalling while setting the CUDA variables and it fixed the issue. Thanks for the help!

[szhang99-bu]

@egurapha Running into the same issue currently, where is the CUDA_HOME and other cuda environment varibale seted?

[ramithuh]

@szhang99-bu,

If you are using CUDA through a conda environment install like (conda install nvidia/label/cuda-11.3.1::cuda), you might need to have these env variables set , so that it correctly points to the environment's CUDA directories.

Need to have these before installing openfold, as @jozhang97 pointed out.

export CUDA_HOME=$CONDA_PREFIX
export CUDA_PATH=$CONDA_PREFIX
export LIBRARY_PATH=$CONDA_PREFIX/lib:$LIBRARY_PATH
export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH