CLIP_image_ViT-L_14 image encoder unequal dimensions.

[Leterax]

After converting the L_14 model using python3 convert_clip.py --model ViT-L/14 --image_output models/CLIP_image_ViT-L_14/ and loading it using image_encoder = tf.keras.models.load_model("CLIP_image_ViT-L_14/", compile=False) it fails to embed an image with this error:

In [5]: image_encoder(np.random.random((1, 480, 480 ,3)))
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[5], line 1
----> 1 image_encoder(np.random.random((1, 480, 480 ,3)))

File ~/.local/lib/python3.10/site-packages/keras/utils/traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     67     filtered_tb = _process_traceback_frames(e.__traceback__)
     68     # To get the full stack trace, call:
     69     # `tf.debugging.disable_traceback_filtering()`
---> 70     raise e.with_traceback(filtered_tb) from None
     71 finally:
     72     del filtered_tb

File ~/.local/lib/python3.10/site-packages/tensorflow/python/framework/ops.py:1967, in _create_c_op(graph, node_def, inputs, control_inputs, op_def, extract_traceback)
   1964   c_op = pywrap_tf_session.TF_FinishOperation(op_desc)
   1965 except errors.InvalidArgumentError as e:
   1966   # Convert to ValueError for backwards compatibility.
-> 1967   raise ValueError(e.message)
   1969 # Record the current Python stack trace as the creating stacktrace of this
   1970 # TF_Operation.
   1971 if extract_traceback:

ValueError: Exception encountered when calling layer 'visual' (type VisualTransformer).

Dimensions must be equal, but are 1157 and 257 for '{{node add}} = AddV2[T=DT_FLOAT](concat, add/ReadVariableOp)' with input shapes: [1,1157,1024], [257,1024].

Call arguments received by layer 'visual' (type VisualTransformer):
  • args=('tf.Tensor(shape=(1, 480, 480, 3), dtype=float32)',)
  • kwargs=<class 'inspect._empty'>

However the converting itself throws no error even though this also performs validation.