person-vehicle-bike-detection-2004 model can not be load with latest opencv dldt 2021.4

[fuatka]

I'm using OpenVINO models with opencv dldt support. person-vehicle-bike-detection-2004 model input resolution is widescreen format what I was looking for. Still fast and mAP result looks great.

But I can not load this model with opencv-4.5.3-dldt-2021.4 using readNet function! This model based on MobileNetV2 backbone with ATSS, different than other person-vehicle-bike-detection models backbone.

There is no issue with person-vehicle-bike-detection-2000, person-vehicle-bike-detection-2001, person-vehicle-bike-detection-2002

[vladimir-dudnik]

@fuatka that is possible issue with OpenCV DNN, but you should be still able to use model directly with OpenVINO, is not it? You may try Open Model Zoo Python object_detection_demo to see how model work with OpenVINO directly.

@alvoron please look at this issue from OpenCV point of view

[alvoron]

@fuatka thank you for raising the issue. Could you please share your code to reproduce the issue? It helps OpenCV team to investigate the issue faster.

[AvenSun]

@alvoron

#opencv/opencv/issues/20380 some related problem can not get correct result via OpenCv DNN with some OpenVINO models

[alalek]

There are 2 issues of OpenCV DNN module:

• 1D layouts. OpenCV uses Mat internally which supports 2D+ layouts only (1D is forced as 2D).
• OpenCV DNN module supports FP32 output only. See 'labels' layout. It is with I32 precision. OpenCV DNN tries to reconfigure output precision here, but looks like this doesn't work anymore. Exception message is: [ PARAMETER_MISMATCH ] Failed to set output blob with precision: FP32, if CNNNetwork output blob precision is: I32

[alalek]

with opencv-4.5.3-dldt-2021.4

Could you try OpenCV build of that package from here? (it is built for PR https://github.com/opencv/opencv/pull/20423 which includes PR with fix: https://github.com/opencv/opencv/pull/20410 )

[fuatka]

I tried this precommit build with opencv dnn object_detection.cpp sample code.

This time previous exception [ PARAMETER_MISMATCH ] Failed to set output Blob. Dimensions mismatch doesn't occured, but get some warnings at the same line

[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (974) cv::dnn::ngraphDataOutputNode Reallocate output 'TopK_1123.0' blob
due to wrong dims: 2 => 1
[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (974) cv::dnn::ngraphDataOutputNode Reallocate output 'TopK_1339.0' blob
due to wrong dims: 2 => 1
[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (974) cv::dnn::ngraphDataOutputNode Reallocate output 'TopK_1555.0' blob
due to wrong dims: 2 => 1
[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (974) cv::dnn::ngraphDataOutputNode Reallocate output 'TopK_1829.0' blob
due to wrong dims: 2 => 1
[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (974) cv::dnn::ngraphDataOutputNode Reallocate output 'TopK_691.0' blob
due to wrong dims: 2 => 1
[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (974) cv::dnn::ngraphDataOutputNode Reallocate output 'TopK_907.0' blob
due to wrong dims: 2 => 1
[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (969) cv::dnn::ngraphDataOutputNode Reallocate output 'labels' blob due
to wrong precision: FP32 => I32 ndims=1
[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (974) cv::dnn::ngraphDataOutputNode Reallocate output 'labels' blob due
to wrong dims: 2 => 1

Another issue is object_detection.cpp doesn't have output layer support except from DetectionOutput and Region. So I get "Unknown output layer type: TopK" error at line

[AvenSun]

@alalek I tried the package downloaded from the link above via python. for example person-vehicle-bike-detection-2004

dnn = cv2.dnn.Net_readFromModelOptimizer(xml, bin)

we will get output names via

outputNames = dnn.getUnconnectedOutLayersNames()

====outputNames==== ['TopK_1123.0', 'TopK_1339.0', 'TopK_1555.0', 'TopK_1829.0', 'TopK_691.0', 'TopK_907.0', 'boxes', 'labels'] ====outputNames====

then if we pass the outputNames to forward

blob = cv2.dnn.blobFromImage(frame, 1.0, size, swapRB=False, crop=False)
net.setInput(blob)
outs = net.forward(outputNames)
print('====outs length====')
print(len(outs))
print('====outs length====')

====outs length====
8
====outs length====

generally it works now , but with some warnings

[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (974) cv::dnn::ngraphDataOutputNode Reallocate output 'TopK_1123.0' blob
 due to wrong dims: 2 => 1
[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (974) cv::dnn::ngraphDataOutputNode Reallocate output 'TopK_1339.0' blob
 due to wrong dims: 2 => 1
[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (974) cv::dnn::ngraphDataOutputNode Reallocate output 'TopK_1555.0' blob
 due to wrong dims: 2 => 1
[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (974) cv::dnn::ngraphDataOutputNode Reallocate output 'TopK_1829.0' blob
 due to wrong dims: 2 => 1
[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (974) cv::dnn::ngraphDataOutputNode Reallocate output 'TopK_691.0' blob
due to wrong dims: 2 => 1
[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (974) cv::dnn::ngraphDataOutputNode Reallocate output 'TopK_907.0' blob
due to wrong dims: 2 => 1
[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (969) cv::dnn::ngraphDataOutputNode Reallocate output 'labels' blob due
to wrong precision: FP32 => I32  ndims=1
[ WARN:0] global C:\build\precommit_custom_windows\opencv\modules\dnn\src\ie_ngr
aph.cpp (974) cv::dnn::ngraphDataOutputNode Reallocate output 'labels' blob due
to wrong dims: 2 => 1

as I mentioned in issue opencv/opencv/issues/20380 though we got 8 outputNames, the document page just lists boxes and lables actually we just need these two to get the result