Converting keras model errors out

[ambud]

❓Question

Trying to use coremltools to convert yolov8 keras (from keras cv github readme) but it errors out. https://github.com/keras-team/keras-cv Since I haven't trained or tuned this model, wondering what could be going wrong that is causing this issue. Is the issue specifying output types or will the converter automatically infer it?

model = keras_cv.models.YOLOV8Detector.from_preset(
        "yolo_v8_m_pascalvoc",
        bounding_box_format="xywh",
)

classifier_config = coremltools.ClassifierConfig(class_ids)
coreml_model = coremltools.converters.convert(model, convert_to="mlprogram",
    inputs=[coremltools.ImageType(shape=(1, ROWS, COLS, 3,),
                           bias=[-1,-1,-1], scale=1/127.5)],
    classifier_config=classifier_config
)

File "/Users/xxxxxx/coremltools-venv/lib/python3.11/site-packages/coremltools/converters/mil/mil/ops/defs/iOS15/classify.py", line 74, in type_inference     r
aise ValueError(msg) ValueError: In op 'classify', number of classes must match the size of the tensor corresponding to 'probabilities'.

[TobyRoseman]

Since you're already using the classifier_config parameter, I wouldn't expect the outputs parameter to help you here. However it's probably worth a try.

Are you sure your class_ids is correct? It seems like the conversion process thinks the number probabilities being outputted by the model is different than len(class_id).

If this doesn't help, I can take a deeper look. However, first I'll need code to reproduce the error.

[ambud]

Thank you for the response. I have checked the class_ids seem to be correct since the model is working (20 classes). Below is the code:

Code:

import tensorflow as tf
from tensorflow import keras

import keras_cv
from keras_cv import visualization

import keras_cv
import numpy as np

import coremltools

ROWS=640
COLS=ROWS

class_ids = open("pascalvoc.txt").readlines()
class_mapping = dict(zip(range(len(class_ids)), class_ids))

inference_resizing = keras_cv.layers.Resizing(
        ROWS, COLS, pad_to_aspect_ratio=True, bounding_box_format="xywh"
)
model = keras_cv.models.YOLOV8Detector.from_preset(
        "yolo_v8_m_pascalvoc",
        bounding_box_format="xywh",
        num_classes=len(class_ids)
)

filepath = keras.utils.get_file(origin="https://i.imgur.com/gCNcJJI.jpg")
image = np.array(keras.utils.load_img(filepath))
batch = inference_resizing([image])
predictions = model.predict(batch)

visualization.plot_bounding_box_gallery(
        batch,
        value_range=(0, 255),
        rows=1,
        cols=1,
        line_thickness=1,
        y_pred=predictions,
        scale=5,
        font_scale=0.1,
        bounding_box_format="xywh",
        class_mapping=class_mapping,
    )

# Errors here
classifier_config = coremltools.ClassifierConfig(class_ids)
coreml_model = coremltools.converters.convert(model, convert_to="mlprogram",
    inputs=[coremltools.ImageType(shape=(1, ROWS, COLS, 3,),
                           bias=[-1,-1,-1])],
     classifier_config=classifier_config
    )

Packages

python3 -m pip install tensorflow keras_cv numpy coremltools 

pascalvoc.txt

aeroplane
bicycle
bird
boat
bottle
bus
car
cat
chair
cow
diningtable
dog
horse
motorbike
person
pottedplant
sheep
sofa
train
tvmonitor

[TobyRoseman]

Yeah, something is wrong here. Looking at the code around the assertion, self.probabilities.shape is (1, 8400, 64) while len(self.classes.val) is 20.

[ambud]

Thanks for the triage. Would you be able to share some timeline estimates for the fix?

[ambud]

Hi @TobyRoseman is there any update on the fix for this issue?