Pytorch without GPU

[ticlazau]

Hello,

can this be run on a system without GPU? When running:

export CUDA_VISIBLE_DEVICES=""

import minimal20b
import torch

model = minimal20b.create_model(
    "/home/coredump/projects/gpt-neox/20B_checkpoints/global_step150000",
    use_cache=True,
    device="cpu",
)
tokenizer = minimal20b.create_tokenizer(
    "/home/coredump/projects/gpt-neox/20B_checkpoints/20B_tokenizer.json",
)


with torch.inference_mode():
    minimal20b.greedy_generate_text(
        model, tokenizer,
        "GPTNeoX20B is a 20B-parameter autoregressive Transformer model developed by EleutherAI.",
        max_seq_len=100,
    )

I am getting: " File "/home/coredump/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/cuda/init.py", line 214, in _lazy_init torch._C._cuda_init() RuntimeError: No CUDA GPUs are available"

Rgds, FM