YAD2K
YAD2K copied to clipboard
allanzelener
Tiny Yolo conversion fails to detect any objects
Converting the darknet tiny-yolo.cfg (only modified for number of classes) fails to detect any objects in Keras.
To be clear:
sean@sean-linux:~/src/darknet/cfg$ diff tiny-yolo.cfg modified-tiny-yolo.cfg
114c114
< filters=425
---
> filters=30
118c118
< anchors = 0.738768,0.874946, 2.42204,2.65704, 4.30971,7.04493, 10.246,4.59428, 12.6868,11.8741
---
> anchors = 1.82,0.74, 3.09,1.66, 4.04,2.46, 4.87,3.19, 6.32,4.36
120c120
< classes=80
---
> classes=1
Validating the network in Darknet yields very positive results:
./darknet detector recall ./data/modified.data ./cfg/modified-tiny-yolo.cfg ./backup/modified-tiny-yolo_5000.weights -i 0 -thresh 0.2
layer filters size input output
0 conv 16 3 x 3 / 1 416 x 416 x 3 -> 416 x 416 x 16
1 max 2 x 2 / 2 416 x 416 x 16 -> 208 x 208 x 16
2 conv 32 3 x 3 / 1 208 x 208 x 16 -> 208 x 208 x 32
3 max 2 x 2 / 2 208 x 208 x 32 -> 104 x 104 x 32
4 conv 64 3 x 3 / 1 104 x 104 x 32 -> 104 x 104 x 64
5 max 2 x 2 / 2 104 x 104 x 64 -> 52 x 52 x 64
6 conv 128 3 x 3 / 1 52 x 52 x 64 -> 52 x 52 x 128
7 max 2 x 2 / 2 52 x 52 x 128 -> 26 x 26 x 128
8 conv 256 3 x 3 / 1 26 x 26 x 128 -> 26 x 26 x 256
9 max 2 x 2 / 2 26 x 26 x 256 -> 13 x 13 x 256
10 conv 512 3 x 3 / 1 13 x 13 x 256 -> 13 x 13 x 512
11 max 2 x 2 / 1 13 x 13 x 512 -> 13 x 13 x 512
12 conv 1024 3 x 3 / 1 13 x 13 x 512 -> 13 x 13 x1024
13 conv 1024 3 x 3 / 1 13 x 13 x1024 -> 13 x 13 x1024
14 conv 30 1 x 1 / 1 13 x 13 x1024 -> 13 x 13 x 30
15 detection
Loading weights from ./backup/modified-tiny-yolo_5000.weights...
seen 32
Done!
Learning Rate: 0.001, Momentum: 0.9, Decay: 0.0005
0 1 1 RPs/Img: 1.00 IOU: 79.94% Recall:100.00%
1 2 2 RPs/Img: 1.00 IOU: 79.03% Recall:100.00%
...
440 420 441 RPs/Img: 0.99 IOU: 79.02% Recall:95.24%
441 421 442 RPs/Img: 0.99 IOU: 79.03% Recall:95.25%
Conversion to Keras appears to go without any problems:
(yad2kenv) bash-3.2$ python3 yad2k.py -p ../modified-tiny-yolo.cfg ../modified-tiny-yolo_5000.weights model_data/modified-tiny.h5
Using TensorFlow backend.
/Users/sean/src/YOLO-CoreML-MPSNNGraph/Convert/yad2k/yad2kenv/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
return f(*args, **kwds)
Loading weights.
Weights Header: [ 0 1 0 320000]
Parsing Darknet config.
Creating Keras model.
Parsing section net_0
Parsing section convolutional_0
conv2d bn leaky (3, 3, 3, 16)
2017-11-05 19:40:50.378533: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX
Parsing section maxpool_0
Parsing section convolutional_1
conv2d bn leaky (3, 3, 16, 32)
Parsing section maxpool_1
Parsing section convolutional_2
conv2d bn leaky (3, 3, 32, 64)
Parsing section maxpool_2
Parsing section convolutional_3
conv2d bn leaky (3, 3, 64, 128)
Parsing section maxpool_3
Parsing section convolutional_4
conv2d bn leaky (3, 3, 128, 256)
Parsing section maxpool_4
Parsing section convolutional_5
conv2d bn leaky (3, 3, 256, 512)
Parsing section maxpool_5
Parsing section convolutional_6
conv2d bn leaky (3, 3, 512, 1024)
Parsing section convolutional_7
conv2d bn leaky (3, 3, 1024, 1024)
Parsing section convolutional_8
conv2d linear (1, 1, 1024, 30)
Parsing section region_0
____________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
====================================================================================================
input_1 (InputLayer) (None, 416, 416, 3) 0
____________________________________________________________________________________________________
convolution2d_1 (Convolution2D) (None, 416, 416, 16) 432 input_1[0][0]
____________________________________________________________________________________________________
batchnormalization_1 (BatchNorma (None, 416, 416, 16) 64 convolution2d_1[0][0]
____________________________________________________________________________________________________
leakyrelu_1 (LeakyReLU) (None, 416, 416, 16) 0 batchnormalization_1[0][0]
____________________________________________________________________________________________________
maxpooling2d_1 (MaxPooling2D) (None, 208, 208, 16) 0 leakyrelu_1[0][0]
____________________________________________________________________________________________________
convolution2d_2 (Convolution2D) (None, 208, 208, 32) 4608 maxpooling2d_1[0][0]
____________________________________________________________________________________________________
batchnormalization_2 (BatchNorma (None, 208, 208, 32) 128 convolution2d_2[0][0]
____________________________________________________________________________________________________
leakyrelu_2 (LeakyReLU) (None, 208, 208, 32) 0 batchnormalization_2[0][0]
____________________________________________________________________________________________________
maxpooling2d_2 (MaxPooling2D) (None, 104, 104, 32) 0 leakyrelu_2[0][0]
____________________________________________________________________________________________________
convolution2d_3 (Convolution2D) (None, 104, 104, 64) 18432 maxpooling2d_2[0][0]
____________________________________________________________________________________________________
batchnormalization_3 (BatchNorma (None, 104, 104, 64) 256 convolution2d_3[0][0]
____________________________________________________________________________________________________
leakyrelu_3 (LeakyReLU) (None, 104, 104, 64) 0 batchnormalization_3[0][0]
____________________________________________________________________________________________________
maxpooling2d_3 (MaxPooling2D) (None, 52, 52, 64) 0 leakyrelu_3[0][0]
____________________________________________________________________________________________________
convolution2d_4 (Convolution2D) (None, 52, 52, 128) 73728 maxpooling2d_3[0][0]
____________________________________________________________________________________________________
batchnormalization_4 (BatchNorma (None, 52, 52, 128) 512 convolution2d_4[0][0]
____________________________________________________________________________________________________
leakyrelu_4 (LeakyReLU) (None, 52, 52, 128) 0 batchnormalization_4[0][0]
____________________________________________________________________________________________________
maxpooling2d_4 (MaxPooling2D) (None, 26, 26, 128) 0 leakyrelu_4[0][0]
____________________________________________________________________________________________________
convolution2d_5 (Convolution2D) (None, 26, 26, 256) 294912 maxpooling2d_4[0][0]
____________________________________________________________________________________________________
batchnormalization_5 (BatchNorma (None, 26, 26, 256) 1024 convolution2d_5[0][0]
____________________________________________________________________________________________________
leakyrelu_5 (LeakyReLU) (None, 26, 26, 256) 0 batchnormalization_5[0][0]
____________________________________________________________________________________________________
maxpooling2d_5 (MaxPooling2D) (None, 13, 13, 256) 0 leakyrelu_5[0][0]
____________________________________________________________________________________________________
convolution2d_6 (Convolution2D) (None, 13, 13, 512) 1179648 maxpooling2d_5[0][0]
____________________________________________________________________________________________________
batchnormalization_6 (BatchNorma (None, 13, 13, 512) 2048 convolution2d_6[0][0]
____________________________________________________________________________________________________
leakyrelu_6 (LeakyReLU) (None, 13, 13, 512) 0 batchnormalization_6[0][0]
____________________________________________________________________________________________________
maxpooling2d_6 (MaxPooling2D) (None, 13, 13, 512) 0 leakyrelu_6[0][0]
____________________________________________________________________________________________________
convolution2d_7 (Convolution2D) (None, 13, 13, 1024) 4718592 maxpooling2d_6[0][0]
____________________________________________________________________________________________________
batchnormalization_7 (BatchNorma (None, 13, 13, 1024) 4096 convolution2d_7[0][0]
____________________________________________________________________________________________________
leakyrelu_7 (LeakyReLU) (None, 13, 13, 1024) 0 batchnormalization_7[0][0]
____________________________________________________________________________________________________
convolution2d_8 (Convolution2D) (None, 13, 13, 1024) 9437184 leakyrelu_7[0][0]
____________________________________________________________________________________________________
batchnormalization_8 (BatchNorma (None, 13, 13, 1024) 4096 convolution2d_8[0][0]
____________________________________________________________________________________________________
leakyrelu_8 (LeakyReLU) (None, 13, 13, 1024) 0 batchnormalization_8[0][0]
____________________________________________________________________________________________________
convolution2d_9 (Convolution2D) (None, 13, 13, 30) 30750 leakyrelu_8[0][0]
====================================================================================================
Total params: 15,770,510
Trainable params: 15,764,398
Non-trainable params: 6,112
____________________________________________________________________________________________________
None
Saved Keras model to model_data/modified-tiny.h5
Read 15770510 of 15770510.0 from Darknet weights.
Saved model plot to model_data/modified-tiny.png
But no results are detected when testing the keras model (even against the same images that I can detect in Darknet):
(yad2kenv) bash-3.2$ python3 test_yolo.py model_data/modified-tiny.h5 -a model_data/modified-tiny_anchors.txt -c model_data/modified_classes.txt
Using TensorFlow backend.
/Users/sean/src/YOLO-CoreML-MPSNNGraph/Convert/yad2k/yad2kenv/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
return f(*args, **kwds)
2017-11-05 19:41:17.192237: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX
/Users/sean/src/YOLO-CoreML-MPSNNGraph/Convert/yad2k/yad2kenv/lib/python3.6/site-packages/keras/models.py:150: UserWarning: No training configuration found in save file: the model was *not* compiled. Compile it manually.
warnings.warn('No training configuration found in save file: '
model_data/modified-tiny.h5 model, anchors, and classes loaded.
Found 0 boxes for 031.JPEG
Found 0 boxes for 032.JPEG
Found 0 boxes for 033.JPEG
Found 0 boxes for dog.jpg
Found 0 boxes for eagle.jpg
Found 0 boxes for giraffe.jpg
Found 0 boxes for horses.jpg
Found 0 boxes for person.jpg
Found 0 boxes for modified_1048.jpg
Found 0 boxes for modified_605.jpg
Found 0 boxes for scream.jpg
line 79
yolo_model = load_model(model_path)
to
yolo_model = load_model(model_path, compile=False)
Fixed
@shady831213 Thanks for sharing your solution. Could you tell me which version of tf and keras are you using? I got TypeError: load_model() got an unexpected keyword argument 'compile' when change yolo_model. Thanks! I'm using keras 1.2.2 and tf 1.4.1.
keras2.0.3……tf-gpu，but I forget the tf version…sorry
发自我的 iPhone
在 2018年1月17日，下午2:10，LI Ben [email protected] 写道：
@shady831213 Thanks for sharing your solution. Could you tell me which version of tf and keras are you using? I got TypeError: load_model() got an unexpected keyword argument 'compile' when change yolo_model. Thanks! I'm using keras 1.2.2 and tf 1.4.1.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
@shady831213 Thanks! I fixed that TypeError with keras2.1.3, but still got Found o boxes for xxx.jpg, anybody has other solution? Thanks!
:)
发自我的 iPhone
在 2018年1月17日，下午6:44，LI Ben [email protected] 写道：
@shady831213 Thanks! I fixed that TypeError with keras2.1.3, but still got Found o boxes for xxx.jpg, anybody has other solution? Thanks!
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
Hi, I have the same issue with 0 boxes found. Has anyone resolved this yet?
@shady831213 Is adding compile=False suppose to fix the issue? I get an error when adding the change to test_yolo.py
File "test_yolo.py", line 79, in _main yolo_model = load_model(model_path,compile=False) TypeError: load_model() got an unexpected keyword argument 'compile'
Hi, I have the same issue with 0 boxes found. Has anyone resolved this yet?
I also find 0 boxes using tiny-yolo coco cfg and weights
I change weights_file.read(16)) to weights_file.read(20)) in yad2k.py and it works!
weights_header = np.ndarray(
shape=(4, ), dtype='int32', buffer=weights_file.read(20))
#104
Thax Weijay - that works - change then mentioned code in yad2k.py does the job. Can you explain what this does and how you got to it?
@holger-prause , This is just my opinion, I really don't know for sure. I just look into the source code in darknet to figure out how to load weights.
In parser.c,
void load_weights(network *net, char *filename) {
load_weights_upto(net, filename, 0, net->n);
}
void load_weights_upto(network *net, char *filename, int start, int cutoff) {
#ifdef GPU
if(net->gpu_index >= 0){
cuda_set_device(net->gpu_index);
}
#endif
fprintf(stderr, "Loading weights from %s...", filename);
fflush(stdout);
FILE *fp = fopen(filename, "rb");
if(!fp) file_error(filename);
int major;
int minor;
int revision;
fread(&major, sizeof(int), 1, fp); // sizeof(int): 4
fread(&minor, sizeof(int), 1, fp); // sizeof(int): 4
fread(&revision, sizeof(int), 1, fp); // sizeof(int): 4
if ((major*10 + minor) >= 2 && major < 1000 && minor < 1000){
fread(net->seen, sizeof(size_t), 1, fp); // sizeof(size_t): 8
} else {
int iseen = 0;
fread(&iseen, sizeof(int), 1, fp); // sizeof(int): 4
*net->seen = iseen;
}
...
This is the header part of weight, so I assume it would be 20 bytes (4+4+4+8) in Tiny-YOLO and 16 bytes (4+4+4+4) in YOLO.
Ah ok - thx for the clarification - even with the source code i am not sure i would have come to the same conclusion - well done!
In class YOLO(object): function def init(self) self.model_image_size = (416, 416) Change (416, 416) to (None, None) --> the result is the same as running with darknet If your detected boxes is more than 20, add max_boxes in the function call yolo_eval boxes, scores, classes = yolo_eval(self.yolo_model.output, self.anchors, len(self.class_names), self.input_image_shape,max_boxes = 100, score_threshold=self.score, iou_threshold=self.iou)
Hey brother im facing this issue
Layer (type) Output Shape Param # Connected to
input_1 (InputLayer) (None, None, None, 3 0
conv2d_1 (Conv2D) (None, None, None, 3 864 input_1[0][0]
batch_normalization_1 (BatchNor (None, None, None, 3 128 conv2d_1[0][0]
leaky_re_lu_1 (LeakyReLU) (None, None, None, 3 0 batch_normalization_1[0][0]
max_pooling2d_1 (MaxPooling2D) (None, None, None, 3 0 leaky_re_lu_1[0][0]
conv2d_2 (Conv2D) (None, None, None, 6 18432 max_pooling2d_1[0][0]
batch_normalization_2 (BatchNor (None, None, None, 6 256 conv2d_2[0][0]
leaky_re_lu_2 (LeakyReLU) (None, None, None, 6 0 batch_normalization_2[0][0]
max_pooling2d_2 (MaxPooling2D) (None, None, None, 6 0 leaky_re_lu_2[0][0]
conv2d_3 (Conv2D) (None, None, None, 1 73728 max_pooling2d_2[0][0]
batch_normalization_3 (BatchNor (None, None, None, 1 512 conv2d_3[0][0]
leaky_re_lu_3 (LeakyReLU) (None, None, None, 1 0 batch_normalization_3[0][0]
conv2d_4 (Conv2D) (None, None, None, 6 8192 leaky_re_lu_3[0][0]
batch_normalization_4 (BatchNor (None, None, None, 6 256 conv2d_4[0][0]
leaky_re_lu_4 (LeakyReLU) (None, None, None, 6 0 batch_normalization_4[0][0]
conv2d_5 (Conv2D) (None, None, None, 1 73728 leaky_re_lu_4[0][0]
batch_normalization_5 (BatchNor (None, None, None, 1 512 conv2d_5[0][0]
leaky_re_lu_5 (LeakyReLU) (None, None, None, 1 0 batch_normalization_5[0][0]
max_pooling2d_3 (MaxPooling2D) (None, None, None, 1 0 leaky_re_lu_5[0][0]
conv2d_6 (Conv2D) (None, None, None, 2 294912 max_pooling2d_3[0][0]
batch_normalization_6 (BatchNor (None, None, None, 2 1024 conv2d_6[0][0]
leaky_re_lu_6 (LeakyReLU) (None, None, None, 2 0 batch_normalization_6[0][0]
conv2d_7 (Conv2D) (None, None, None, 1 32768 leaky_re_lu_6[0][0]
batch_normalization_7 (BatchNor (None, None, None, 1 512 conv2d_7[0][0]
leaky_re_lu_7 (LeakyReLU) (None, None, None, 1 0 batch_normalization_7[0][0]
conv2d_8 (Conv2D) (None, None, None, 2 294912 leaky_re_lu_7[0][0]
batch_normalization_8 (BatchNor (None, None, None, 2 1024 conv2d_8[0][0]
leaky_re_lu_8 (LeakyReLU) (None, None, None, 2 0 batch_normalization_8[0][0]
max_pooling2d_4 (MaxPooling2D) (None, None, None, 2 0 leaky_re_lu_8[0][0]
conv2d_9 (Conv2D) (None, None, None, 5 1179648 max_pooling2d_4[0][0]
batch_normalization_9 (BatchNor (None, None, None, 5 2048 conv2d_9[0][0]
leaky_re_lu_9 (LeakyReLU) (None, None, None, 5 0 batch_normalization_9[0][0]
conv2d_10 (Conv2D) (None, None, None, 2 131072 leaky_re_lu_9[0][0]
batch_normalization_10 (BatchNo (None, None, None, 2 1024 conv2d_10[0][0]
leaky_re_lu_10 (LeakyReLU) (None, None, None, 2 0 batch_normalization_10[0][0]
conv2d_11 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_10[0][0]
batch_normalization_11 (BatchNo (None, None, None, 5 2048 conv2d_11[0][0]
leaky_re_lu_11 (LeakyReLU) (None, None, None, 5 0 batch_normalization_11[0][0]
conv2d_12 (Conv2D) (None, None, None, 2 131072 leaky_re_lu_11[0][0]
batch_normalization_12 (BatchNo (None, None, None, 2 1024 conv2d_12[0][0]
leaky_re_lu_12 (LeakyReLU) (None, None, None, 2 0 batch_normalization_12[0][0]
conv2d_13 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_12[0][0]
batch_normalization_13 (BatchNo (None, None, None, 5 2048 conv2d_13[0][0]
how to fix it?? please help me out.
A couple of solutions I have found for allowing both yolo and yolo-tiny to load:
First simple, non-robust method using the name of the file:
bytes_len = 20 if '-tiny' in args.weights_path else 16
weights_header = np.array(np.ndarray(
shape=(4, ), dtype='int32', buffer=weights_file.read(16)))
Second method, from the darknet code found by @Willjay90 👍
weights_header = np.int64(np.ndarray(
shape=(4, ), dtype='int32', buffer=weights_file.read(16)))
major = weights_header[0]
minor = weights_header[1]
if (major*10 + minor) >= 2 and major < 1000 and minor < 1000:
final = np.int64(np.ndarray(shape=(1, ), dtype='int32', buffer=weights_file.read(4)))
weights_header[3] = np.bitwise_or(np.left_shift(final, 4), weights_header[3])
print('Weights Header: ', weights_header)
