CLIP-tf2
CLIP-tf2 copied to clipboard
CLIP_image_ViT-L_14 image encoder unequal dimensions.
After converting the L_14 model using python3 convert_clip.py --model ViT-L/14 --image_output models/CLIP_image_ViT-L_14/
and loading it using image_encoder = tf.keras.models.load_model("CLIP_image_ViT-L_14/", compile=False)
it fails to embed an image with this error:
In [5]: image_encoder(np.random.random((1, 480, 480 ,3)))
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[5], line 1
----> 1 image_encoder(np.random.random((1, 480, 480 ,3)))
File ~/.local/lib/python3.10/site-packages/keras/utils/traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
67 filtered_tb = _process_traceback_frames(e.__traceback__)
68 # To get the full stack trace, call:
69 # `tf.debugging.disable_traceback_filtering()`
---> 70 raise e.with_traceback(filtered_tb) from None
71 finally:
72 del filtered_tb
File ~/.local/lib/python3.10/site-packages/tensorflow/python/framework/ops.py:1967, in _create_c_op(graph, node_def, inputs, control_inputs, op_def, extract_traceback)
1964 c_op = pywrap_tf_session.TF_FinishOperation(op_desc)
1965 except errors.InvalidArgumentError as e:
1966 # Convert to ValueError for backwards compatibility.
-> 1967 raise ValueError(e.message)
1969 # Record the current Python stack trace as the creating stacktrace of this
1970 # TF_Operation.
1971 if extract_traceback:
ValueError: Exception encountered when calling layer 'visual' (type VisualTransformer).
Dimensions must be equal, but are 1157 and 257 for '{{node add}} = AddV2[T=DT_FLOAT](concat, add/ReadVariableOp)' with input shapes: [1,1157,1024], [257,1024].
Call arguments received by layer 'visual' (type VisualTransformer):
• args=('tf.Tensor(shape=(1, 480, 480, 3), dtype=float32)',)
• kwargs=<class 'inspect._empty'>
However the converting itself throws no error even though this also performs validation.