Execution on CPU it's generating error by datatype half (float16) in model CLIP

[JohanValero]

Running the repository in Torch CPU.

In mdm.py for the class MDM in the method "load_and_freeze_clip".

def load_and_freeze_clip(self, clip_version):
    clip_model, clip_preprocess = clip.load(clip_version, device='cpu',
                                            jit=False)  # Must set jit=False for training
    clip.model.convert_weights(
        clip_model)  # Actually this line is unnecessary since clip by default already on float16.

The comment solved the problem, but in the class Transformer init ResidualAttentionBlock continue with default float16. Don't considering the inference on cpu.