Training balloon model on gpu: AttributeError: 'Model' object has no attribute 'metrics_tensors'

[HjalteMann]

I am trying to train a model with the balloon example. I can get the training running on my local cpu, but when I attempt to run the balloon.py on my gpu (and actually also from the Jupyter Notebook) I get the error below. Everything seemed to install fine, bot requirements and the setup.py.

python balloon.py train dataset=/home/Mask_RCNN/samples/balloon/balloon/ --weights=mask_rcnn_coco.h5

Traceback (most recent call last): File "/home/bitcue/Dropbox/mrcnn2/Mask_RCNN/samples/balloon/balloon.py", line 367, in <module> train(model) File "/home/bitcue/Dropbox/mrcnn2/Mask_RCNN/samples/balloon/balloon.py", line 201, in train layers='heads') File "/home/bitcue/anaconda3/envs/mrcnn2/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 2354, in train File "/home/bitcue/anaconda3/envs/mrcnn2/lib/python3.6/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py", line 2199, in compile AttributeError: 'Model' object has no attribute 'metrics_tensors

I have the following packages installed in an anaconda environment:

Name Version Build Channel _libgcc_mutex 0.1 main _tflow_select 2.1.0 gpu absl-py 0.7.1 py36_0 astor 0.8.0 py36_0 backports 1.0 py_2 backports.weakref 1.0rc1 py36_0 blas 1.0 mkl bleach 1.5.0 py36_0 c-ares 1.15.0 h7b6447c_1001 ca-certificates 2019.5.15 1 anaconda certifi 2019.6.16 py36_1 anaconda cudatoolkit 9.0 h13b8566_0 cudnn 7.6.0 cuda9.0_0 cupti 9.0.176 0 expat 2.2.6 he6710b0_0 gast 0.2.2 py36_0 git 2.20.1 pl526hacde149_0 grpcio 1.16.1 py36hf8bcb03_1 h5py 2.9.0 py36h7918eee_0 hdf5 1.10.4 hb1b8bf9_0 html5lib 0.9999999 py36_0 intel-openmp 2019.5 281 keras-applications 1.0.8 py_0 keras-gpu 2.0.8 py36h0585f72_0 anaconda keras-preprocessing 1.1.0 py_1 krb5 1.16.1 h173b8e3_7 libcurl 7.65.3 h20c2e04_0 libedit 3.1.20181209 hc058e9b_0 libffi 3.2.1 hd88cf55_4 libgcc 7.2.0 h69d50b8_2 libgcc-ng 9.1.0 hdf63c60_0 libgfortran-ng 7.3.0 hdf63c60_0 libprotobuf 3.8.0 hd408876_0 libssh2 1.8.2 h1ba5d50_0 libstdcxx-ng 9.1.0 hdf63c60_0 markdown 3.1.1 py36_0 mkl 2019.5 281 mkl-service 2.3.0 py36he904b0f_0 mkl_fft 1.0.14 py36ha843d7b_0 mkl_random 1.0.2 py36hd81dba3_0 mock 3.0.5 py36_0 ncurses 6.1 he6710b0_1 numpy 1.16.5 py36h7e9f1db_0 numpy-base 1.16.5 py36hde5b4d6_0 openssl 1.1.1 h7b6447c_0 anaconda perl 5.26.2 h14c3975_0 pip 19.2.2 py36_0 protobuf 3.8.0 py36he6710b0_0 python 3.6.9 h265db76_0 pyyaml 5.1.2 py36h7b6447c_0 readline 7.0 h7b6447c_5 scipy 1.3.1 py36h7c811a0_0 setuptools 41.0.1 py36_0 six 1.12.0 py36_0 sqlite 3.29.0 h7b6447c_0 tensorboard 1.13.1 py36hf484d3e_0 tensorflow 1.13.1 gpu_py36h26cf82e_0 tensorflow-base 1.13.1 gpu_py36h8f37b9b_0 tensorflow-estimator 1.13.0 py_0 tensorflow-gpu 1.13.1 h0d30ee6_0 tensorflow-gpu-base 1.7.0 py36hcdda91b_1 tensorflow-tensorboard 1.5.1 py36hf484d3e_1 termcolor 1.1.0 py36_1 tk 8.6.8 hbc83047_0 werkzeug 0.15.5 py_0 wheel 0.33.4 py36_0 xz 5.2.4 h14c3975_4 yaml 0.1.7 had09818_2 zlib 1.2.11 h7b6447c_3

[yangdeqieyingli]

i meet the same question,How to solve it?

[msrkp]

Initialise self.keras_model.metrics_tensors = [] then you are good to go...

[Hunain-Gillani]

If anyone is looking for the solution, downgrade keras to 2.1.0. It started working for me.

[HjalteMann]

Thank you for the reply, @51r1u5 ! Where in the scripts should I do this?

@Hunain-G, I'm using keras 2.0.8 as advised in the readme. Do you propose to upgrade?

[Hunain-Gillani]

@HjalteMann , if you're using keras 2.0.8 then there shouldn't have been any issue. I was using keras 2.3.0 and by downgrading keras to 2.1.0 made the issue resolved.

[mashuai191]

since Keras 2.3.0 there is such issue, when i downgrade to 2.2.5 the issue resolved so i believe any version <= 2.2.5 would work, and of course should be >= 2.0.8 as requiremets.txt suggested.

[mffigueroa]

What about changing: self.keras_model.metrics_tensors.append(loss) to: self.keras_model.add_metric(loss, name)

[JepsonNomad]

I got the same error but when trying to run train_shapes (as described here). Fixed by installing keras version 2.2.5 and tensorflow version 1.15.0rc2 (pip3 install keras==2.2.5 and the same formatting for TF). Keras version 2.1.0 did not work for me, nor did version 2.3.0.

[freitaucher]

Did someone succeed to run any example with tensorflow 1.13 ? This is the oldest version which is compatible with cuda10.0. Older versions require older cuda's, but installing several cuda's is "highly" inconvenient

[julian-vidal]

I was having the same error, but after installing keras 2.0.8 as you said above. Now I have other error: File "/Users/julianvidal/anaconda3/envs/MaskRCNN/lib/python3.6/site-packages/keras/callbacks/callbacks.py", line 23, in <module> from ..engine.training_utils import standardize_input_data ModuleNotFoundError: No module named 'keras.engine.training_utils'

Can you help please? Thanks!

[julian-vidal]

Initialise self.keras_model.metrics_tensors = [] then you are good to go...

Can you give more details about how to do this? Thanks!

[rteuwens]

since Keras 2.3.0 there is such issue, when i downgrade to 2.2.5 the issue resolved so i believe any version <= 2.2.5 would work, and of course should be >= 2.0.8 as requiremets.txt suggested.

ran tf.keras.__version__ in the console in a Kaggle kernel and got '2.2.4-tf'. I'm struggling with the same issue, though, and I'm not sure how to downgrade on Kaggle.

[trialzuki]

What about changing: self.keras_model.metrics_tensors.append(loss) to: self.keras_model.add_metric(loss, name)

It's solve the problem for 2.3.0

[HjalteMann]

My issue was solved by installing all dependencies in a new conda environment, with pip, not conda.

[dfy888]

What about changing: self.keras_model.metrics_tensors.append(loss) to: self.keras_model.add_metric(loss, name)

it is work! 非常感谢

[pipun-gif]

What about changing: self.keras_model.metrics_tensors.append(loss) to: self.keras_model.add_metric(loss, name)

it is work! 非常感谢

hi excuse me @dfy888 how to edit model.py inside egg file? i can open it by rename the .egg into .zip and modify all file inside that egg, but unfortunately i don't know package it back as a python egg file after my modification. thank you

[ZOUYIyi]

What about changing: self.keras_model.metrics_tensors.append(loss) to: self.keras_model.add_metric(loss, name)

it is work! 非常感谢

我应该在哪里修改这一句呢？

[pipun-gif]

What about changing: self.keras_model.metrics_tensors.append(loss) to: self.keras_model.add_metric(loss, name) it is work! 非常感谢

我应该在哪里修改这一句呢？

around here : anaconda3/envs/mrcnn2/lib/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py

[oktemerbay]

What about changing: self.keras_model.metrics_tensors.append(loss) to: self.keras_model.add_metric(loss, name) it is work! 非常感谢

我应该在哪里修改这一句呢？

around here : anaconda3/envs/mrcnn2/lib/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py

hi , first change model.py file's line 2199 , which is "self.keras_model.metrics_tensors.append(loss)" to "self.keras_model.add_metric(loss, name)" and then with this command "python .\setup.py install" generate new egg file. You cannot directly change the content of the egg file.

Also , I do not recommend downgrading tensorflow because in return , it generates performance issuses.

[tourist-C]

What about changing: self.keras_model.metrics_tensors.append(loss) to: self.keras_model.add_metric(loss, name) it is work! 非常感谢

我应该在哪里修改这一句呢？

mrcnn\model.py line2199

[abhishekpundir22]

i have keras version 2.2.0 and tf 1.16.0 python 3.6.10 and having this issue.... many said it will work on this... anyone has its fix? I tried multiple combinations of version till now.. nothing worked. thanks!

[rteuwens]

i have keras version 2.2.0 and tf 1.16.0 python 3.6.10 and having this issue.... many said it will work on this... anyone has its fix? I tried multiple combinations of version till now.. nothing worked. thanks!

Hey Abhishek, check out @mffigueroa 's answer:

What about changing: self.keras_model.metrics_tensors.append(loss) to: self.keras_model.add_metric(loss, name)

As @tourist-C points out, it's on line 2199 in mcrnn/model.py As here: https://github.com/rteuwens/Mask_RCNN/blob/master/mrcnn/model.py#L2199

[banafsh89]

What about changing: self.keras_model.metrics_tensors.append(loss) to: self.keras_model.add_metric(loss, name) it is work! 非常感谢

我应该在哪里修改这一句呢？

around here : anaconda3/envs/mrcnn2/lib/site-packages/mask_rcnn-2.1-py3.6.egg/mrcnn/model.py

This path doesn't exist for me. How can I find where the model.py is or where my mrcnn package is saved in anaconda?

[Independent-Laboratory]

In my case, this works.

cd Mask_RCNN # Root directory of Mask RCNN emacs mrcnn/model.py # Edit model.py #-------------------------------------- At line 2199 # self.keras_model.metrics_tensors.append(loss) # <- comment out self.keras_model.add_metric(loss, name) # <- Add #-------------------------------------- sudo python3 setup.py install # <- mask_rcnn-2.1-py3.6.egg is automaticalliy replaced

Then, balloon training script will work.

[Mstfakts]

If anyone is looking for the solution, downgrade keras to 2.1.0. It started working for me.

It did not work for me, I got another error about keras.

[Hunain-Gillani]

If anyone is looking for the solution, downgrade keras to 2.1.0. It started working for me.

It did not work for me, I got another error about keras.

Can you specify the error? What the error is now?

[Mstfakts]

If anyone is looking for the solution, downgrade keras to 2.1.0. It started working for me.

It did not work for me, I got another error about keras.

Can you specify the error? What the error is now?

Actually, I do not exactly remember the error but @mffigueroa 's solution worked well.

[gethubwy]

Initialise self.keras_model.metrics_tensors = [] then you are good to go... well,I use keras version=2.3.1,it is not work for me.

[abhishekpundir22]

I have solved this issue by changing keras version to 2.1.0 and tensorflow version to 1.15.0

[abhishekpundir22]

i have keras version 2.2.0 and tf 1.16.0 python 3.6.10 and having this issue.... many said it will work on this... anyone has its fix? I tried multiple combinations of version till now.. nothing worked. thanks!

Hey Abhishek, check out @mffigueroa 's answer:

What about changing: self.keras_model.metrics_tensors.append(loss) to: self.keras_model.add_metric(loss, name)

As @tourist-C points out, it's on line 2199 in mcrnn/model.py As here: https://github.com/rteuwens/Mask_RCNN/blob/master/mrcnn/model.py#L2199

This works too. I can confirm. Thanks.