RuntimeError: Current CUDA Device does not support bfloat16. Please switch dtype to float16.

[ThibaultGROUEIX]

Hi !

I am checking out BLIP-2 and trying to caption some images with generate.

I get an error :

  caption = model.generate({"image": image.float()}, use_nucleus_sampling=True, num_captions=3)
  File "/home/groueix/.local/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/lavis/models/blip2_models/blip2_t5.py", line 207, in generate
    with torch.amp.autocast(
  File "/home/groueix/.local/lib/python3.8/site-packages/torch/amp/autocast_mode.py", line 225, in __init__
    raise RuntimeError('Current CUDA Device does not support bfloat16. Please switch dtype to float16.')
RuntimeError: Current CUDA Device does not support bfloat16. Please switch dtype to float16.

I am not sure how to fix this, are you able to help?

Thanks :)

Thibault