Get the following error when trying to JIT CoCa

[leizaf]

model, preproc = open_clip.create_model_from_pretrained(
    'coca_ViT-L-14',
    pretrained='laion2b_s13b_b90k',
    device='cuda',
    jit=True
)

RuntimeError: 
'NoneType' object has no attribute or method 'dtype'.:
  File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/activation.py", line 1117
        elif self.in_proj_bias is not None and query.dtype != self.in_proj_bias.dtype:
            why_not_fast_path = f"dtypes of query ({query.dtype}) and self.in_proj_bias ({self.in_proj_bias.dtype}) don't match"
        elif self.in_proj_weight is not None and query.dtype != self.in_proj_weight.dtype:
                                                                ~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
            # this case will fail anyway, but at least they'll get a useful error message.
            why_not_fast_path = f"dtypes of query ({query.dtype}) and self.in_proj_weight ({self.in_proj_weight.dtype}) don't match"

[gpucce]

Hi, @leifu1128 if I remember correctly it is related to this issue https://github.com/pytorch/pytorch/issues/71470