coremltools
coremltools copied to clipboard
Published
20 hours ago •
apple
apple
How to print model layers shape
❓Question
How to print model layers shape
Use neural_network.NeuralNetworkBuilder built a model spec.
when use spec new MLModel produce an error:
"compiler error: Espresso exception: "Invalid blob shape": generic_elementwise_kernel: cannot broadcast [1, 14280, 2, 1, 1] and [14280, 2, 1, 1, 1]".
print spec just some layer connection. no shape info.
I'm using coremltools build a model decode bbox like sdd-caffe does.
System Information
- Mac OS
- coremltools 4.0b2
Code
fd_mbox_priorbox = np.load('fd_mbox_priorbox.npy')
num_prior_ = int(fd_mbox_priorbox.shape[2] / 4)
# MLMultiArray inputs of neural networks must have 1 or 3 dimensions.
# We only have 2, so add an unused dimension of size one at the back.
input_features = [("scores", datatypes.Array(num_classes + 1, num_prior_, 1)),
("boxes", datatypes.Array(4, num_prior_, 1))]
# The outputs of the decoder model should match the inputs of the next
# model in the pipeline, NonMaximumSuppression. This expects the number
# of bounding boxes in the first dimension.
output_features = [
# ("decoded_bbox_out", datatypes.Array(4, num_prior_, 1)),
("confidence", datatypes.Array(num_prior_, num_classes + 1, 1)),
("coordinates", datatypes.Array(num_prior_, 4, 1))
]
builder = neural_network.NeuralNetworkBuilder(input_features,
output_features,
disable_rank5_shape_mapping=True
)
# decode_bbox_center_x = prior_variance[0] * bbox.xmin() * prior_width + prior_center_x;
# decode_bbox_center_y = prior_variance[1] * bbox.ymin() * prior_height + prior_center_y;
# decode_bbox_width = exp(prior_variance[2] * bbox.xmax()) * prior_width;
# decode_bbox_height = exp(prior_variance[3] * bbox.ymax()) * prior_height;
fd_mbox_priorbox = fd_mbox_priorbox.reshape([4, -1])
prior_bbox = fd_mbox_priorbox[:, :num_prior_]
prior_variances = fd_mbox_priorbox[:, num_prior_:]
xmin, ymin, xmax, ymax = prior_bbox
prior_width = xmax - xmin
prior_height = ymax - ymin
prior_center_x = (xmin + xmax) / 2.
prior_center_y = (ymin + ymax) / 2.
prior_wh = np.stack([prior_width, prior_height])
prior_xy = np.stack([prior_center_x, prior_center_y])
prior_variances = np.expand_dims(prior_variances, axis=-1)
prior_wh = np.expand_dims(prior_wh, axis=-1)
prior_xy = np.expand_dims(prior_xy, axis=-1)
builder.add_load_constant('prior_wh',
output_name='prior_wh_out',
constant_value=prior_wh,
shape=prior_wh.shape
)
builder.add_load_constant('prior_xy',
output_name='prior_xy_out',
constant_value=prior_xy,
shape=prior_xy.shape)
builder.add_load_constant('prior_variances',
output_name='prior_variances_out',
constant_value=prior_variances,
shape=prior_variances.shape)
print(prior_variances.shape)
print(prior_wh.shape)
print(prior_xy.shape)
builder.add_split('split_variances',
input_name='prior_variances_out',
output_names=['split_variances_xy', 'split_variances_wh'], # (2,num_prior_,1) , (2,num_prior_,1)
# num_splits=2,
# axis=-3
)
builder.add_split('split_box',
input_name='boxes',
output_names=['split_box_min_out', 'split_box_max_out'], # (2,num_prior_,1) , (2,num_prior_,1)
# num_splits=2,
# axis=-3
)
# prior_variance[0] * bbox.xmin() * prior_width
# prior_variance[1] * bbox.ymin() * prior_height
builder.add_elementwise('bbox_xy_m',
input_names=['split_variances_xy', 'split_box_min_out', 'prior_wh_out'],
output_name='bbox_xy_m_out', # (2,num_prior_,1)
mode='MULTIPLY'
)
# prior_variance[0] * bbox.xmin() * prior_width + prior_center_x
# prior_variance[1] * bbox.ymin() * prior_height + prior_center_y
builder.add_elementwise('bbox_xy_a',
input_names=['bbox_xy_m_out', 'prior_xy_out'],
output_name='bbox_xy_a_out',
mode='ADD'
)
# prior_variance[2] * bbox.xmax()
# prior_variance[3] * bbox.ymax()
builder.add_elementwise('bbox_wh_m',
input_names=['split_variances_wh', 'split_box_max_out'],
output_name='bbox_wh_m_out',
mode='MULTIPLY'
)
# exp(prior_variance[2] * bbox.xmax())
# exp(prior_variance[3] * bbox.ymax())
builder.add_unary('bbox_wh_e',
input_name='bbox_wh_m_out',
output_name='bbox_wh_e_out',
mode='exp'
)
# exp(prior_variance[2] * bbox.xmax()) * prior_width
# exp(prior_variance[3] * bbox.ymax()) * prior_height
builder.add_elementwise('bbox_wh_m_m',
input_names=['bbox_wh_e_out', 'prior_wh_out'],
output_name='bbox_wh_m_m_out',
mode='MULTIPLY'
)
builder.add_elementwise('decoded_bbox',
input_names=['bbox_xy_a_out', 'bbox_wh_m_m_out'],
output_name='decoded_bbox_out',
mode='CONCAT'
)
builder.add_nms('nms',
input_names=['decoded_bbox_out', 'scores'],
output_names=['coordinates', 'confidence', 'nms_indices', 'nms_number'],
iou_threshold=0.3,
score_threshold=0.05,
max_boxes=200
)
builder.inspect_layers(verbose=True)
nms_model = ct.models.MLModel(builder.spec)
nms_model.save('nn.mlmodel')
print("success")
Not sure whether it helps but have you tried logging like in #818 to gain further insights ?
use CoreML::Model load spec with some layer unsupport modification. I can print output blobs shape.
all seems OK .
when use Apple MLModel::compileModelAtURL still produce error
error: Error compiling model: "compiler error: Espresso exception: "Invalid state": generic_split_kernel: Number of top blobs in split kernel does not evenly divide width axis."
error message is so vague. And no way show verbose info about layers output/input shape when compile model.😭
builder.inspect_layers message here. asking for help
/Users/rqg/Playground/coremltools/envs/coremltools-py3.7/bin/python3.7 /Users/rqg/PycharmProjects/CoreML/Convertfd.py
4.0b3
(4, 14280, 1)
(2, 14280, 1)
(2, 14280, 1)
[Id: 11], Name: nms (Type: NonMaximumSuppression)
Updatable: False
Input blobs: ['decoded_bbox_out', 'scores']
Output blobs: ['coordinates', 'confidence', 'nms_indices', 'nms_number']
Parameters:
iouThreshold = 0.30000001192092896
scoreThreshold = 0.05000000074505806
maxBoxes = 200
perClassSuppression = False
[Id: 10], Name: decoded_bbox (Type: concat)
Updatable: False
Input blobs: ['bbox_xy_a_out', 'bbox_wh_m_m_out']
Output blobs: ['decoded_bbox_out']
Parameters:
sequenceConcat = False
[Id: 9], Name: bbox_wh_m_m (Type: multiply)
Updatable: False
Input blobs: ['bbox_wh_e_out', 'prior_wh_out']
Output blobs: ['bbox_wh_m_m_out']
Parameters:
alpha = 0.0
[Id: 8], Name: bbox_wh_e (Type: unary)
Updatable: False
Input blobs: ['bbox_wh_m_out']
Output blobs: ['bbox_wh_e_out']
Parameters:
type = 4
alpha = 1.0
epsilon = 9.999999974752427e-07
shift = 0.0
scale = 1.0
[Id: 7], Name: bbox_wh_m (Type: multiply)
Updatable: False
Input blobs: ['split_variances_wh', 'split_box_max_out']
Output blobs: ['bbox_wh_m_out']
Parameters:
alpha = 0.0
[Id: 6], Name: bbox_xy_a (Type: add)
Updatable: False
Input blobs: ['bbox_xy_m_out', 'prior_xy_out']
Output blobs: ['bbox_xy_a_out']
Parameters:
alpha = 0.0
[Id: 5], Name: bbox_xy_m (Type: multiply)
Updatable: False
Input blobs: ['split_variances_xy', 'split_box_min_out', 'prior_wh_out']
Output blobs: ['bbox_xy_m_out']
Parameters:
alpha = 0.0
[Id: 4], Name: split_box (Type: split)
Updatable: False
Input blobs: ['boxes']
Output blobs: ['split_box_min_out', 'split_box_max_out']
Parameters:
nOutputs = 2
[Id: 3], Name: split_variances (Type: split)
Updatable: False
Input blobs: ['prior_variances_out']
Output blobs: ['split_variances_xy', 'split_variances_wh']
Parameters:
nOutputs = 2
[Id: 2], Name: prior_variances (Type: loadConstant)
Updatable: False
Input blobs: []
Output blobs: ['prior_variances_out']
Parameters:
shape = [4, 14280, 1]
data = (57120 floatValues)
[Id: 1], Name: prior_xy (Type: loadConstant)
Updatable: False
Input blobs: []
Output blobs: ['prior_xy_out']
Parameters:
shape = [2, 14280, 1]
data = (28560 floatValues)
[Id: 0], Name: prior_wh (Type: loadConstant)
Updatable: False
Input blobs: []
Output blobs: ['prior_wh_out']
Parameters:
shape = [2, 14280, 1]
data = (28560 floatValues)
/Users/rqg/Playground/coremltools/coremltools/models/model.py:119: RuntimeWarning: You will not be able to run predict() on this Core ML model. Underlying exception message was: Error compiling model: "compiler error: Espresso exception: "Invalid state": generic_split_kernel: Number of top blobs in split kernel does not evenly divide width axis.".
RuntimeWarning,
Was hunting down for a solution when I got here. One way to do it seems to be rewriting this Swift function in Python:
https://github.com/danielnugraha/flower-ios/blob/21cf129026861d6253030c512647918f7aa6d373/FlowerIOS/Utils/CoreMLHelpers/MLModelInspect.swift#L23-L47
