Error when running main.py

[tszngai]

Hi Fabiocarrara, thank you for your efforts in this incredible project. As I am following with the instructions, I have encountered some errors when running the main.py.

According to the other two posts, I have changed the caffe path to the correct one and remove the GPU 0 flags already. When I run main.py, I got the following errors:

py_lab@Pie:~/deep-parking$ python main.py INFO 2018-07-09 17:16:03: Setting up mAlexNet trained on PKLot_train, validated on PKLot_val, CNRPark-EXT_val, tested on PKLot_test_CNRPark-EXT_test ... INFO 2018-07-09 17:16:04: Training on PKLot_train while validating on PKLot_val, CNRPark-EXT_val ... E0709 17:16:04.984817 4622 io.cpp:80] Could not open or find file /home/py_lab/deep-parking/PKLot/PKLotUFPR04/Sunny/2012-12-08/Empty/2012-12-08_14_25_10#010.jpg F0709 17:16:04.985540 4622 image_data_layer.cpp:77] Check failed: cv_img.data Could not load UFPR04/Sunny/2012-12-08/Empty/2012-12-08_14_25_10#010.jpg *** Check failure stack trace: *** @ 0x7eff5db9f5cd google::LogMessage::Fail() @ 0x7eff5dba1433 google::LogMessage::SendToLog() @ 0x7eff5db9f15b google::LogMessage::Flush() @ 0x7eff5dba1e1e google::LogMessageFatal::~LogMessageFatal() @ 0x7eff5e07b358 caffe::ImageDataLayer<>::DataLayerSetUp() @ 0x7eff5dfe598e caffe::BasePrefetchingDataLayer<>::LayerSetUp() @ 0x7eff5df47d07 caffe::Net<>::Init() @ 0x7eff5df4a44e caffe::Net<>::Net() @ 0x7eff5e0d44fa caffe::Solver<>::InitTrainNet() @ 0x7eff5e0d59c5 caffe::Solver<>::Init() @ 0x7eff5e0d5cdf caffe::Solver<>::Solver() @ 0x7eff5e0cd801 caffe::Creator_SGDSolver<>() @ 0x40a288 train() @ 0x407208 main @ 0x7eff5c335830 __libc_start_main @ 0x407ad9 _start @ (nil) (unknown)

It says that it could not find the file at /home/py_lab/deep-parking/PKLot/PKLotUFPR04/Sunny/2012-12-08/Empty/2012-12-08_14_25_10#010.jpg The problem is that a '/' is missing before UFPR04. UFPR04 is actually a folder within PKLot so it should instead be home/py_lab/deep-parking/PKLot/PKLot/UFPR04/Sunny/2012-12-08/Empty/2012-12-08_14_25_10#010.jpg

Could you please advice how to fix this issue?

[tszngai]

I figured it out... It was because I did not end the path with '/' in the config.py files. Now I fixed it and it still could find the path...

py_lab@Pie:~/deep-parking$ python main.py INFO 2018-07-09 18:57:21: Setting up mAlexNet trained on PKLot_train, validated on PKLot_val, CNRPark-EXT_val, tested on PKLot_test_CNRPark-EXT_test ... INFO 2018-07-09 18:57:22: Training on PKLot_train while validating on PKLot_val, CNRPark-EXT_val ... E0709 18:57:23.132761 5623 io.cpp:80] Could not open or find file /home/py_lab/deep-parking/PKLot/PKLot/UFPR04/Sunny/2012-12-08/Empty/2012-12-08_14_25_10#010.jpg F0709 18:57:23.132954 5623 image_data_layer.cpp:77] Check failed: cv_img.data Could not load UFPR04/Sunny/2012-12-08/Empty/2012-12-08_14_25_10#010.jpg *** Check failure stack trace: *** @ 0x7f4d63fde5cd google::LogMessage::Fail() @ 0x7f4d63fe0433 google::LogMessage::SendToLog() @ 0x7f4d63fde15b google::LogMessage::Flush() @ 0x7f4d63fe0e1e google::LogMessageFatal::~LogMessageFatal() @ 0x7f4d644ba358 caffe::ImageDataLayer<>::DataLayerSetUp() @ 0x7f4d6442498e caffe::BasePrefetchingDataLayer<>::LayerSetUp() @ 0x7f4d64386d07 caffe::Net<>::Init() @ 0x7f4d6438944e caffe::Net<>::Net() @ 0x7f4d645134fa caffe::Solver<>::InitTrainNet() @ 0x7f4d645149c5 caffe::Solver<>::Init() @ 0x7f4d64514cdf caffe::Solver<>::Solver() @ 0x7f4d6450c801 caffe::Creator_SGDSolver<>() @ 0x40a288 train() @ 0x407208 main @ 0x7f4d62774830 __libc_start_main @ 0x407ad9 _start @ (nil) (unknown) INFO 2018-07-09 18:57:23: Setting up AlexNet trained on PKLot_train, validated on PKLot_val, CNRPark-EXT_val, tested on PKLot_test_CNRPark-EXT_test ... INFO 2018-07-09 18:57:24: Training on PKLot_train while validating on PKLot_val, CNRPark-EXT_val ... E0709 18:57:24.787290 5637 io.cpp:80] Could not open or find file /home/py_lab/deep-parking/PKLot/PKLot/UFPR04/Sunny/2012-12-08/Empty/2012-12-08_14_25_10#010.jpg F0709 18:57:24.787415 5637 image_data_layer.cpp:77] Check failed: cv_img.data Could not load UFPR04/Sunny/2012-12-08/Empty/2012-12-08_14_25_10#010.jpg *** Check failure stack trace: *** @ 0x7f8efe7d45cd google::LogMessage::Fail() @ 0x7f8efe7d6433 google::LogMessage::SendToLog() @ 0x7f8efe7d415b google::LogMessage::Flush() @ 0x7f8efe7d6e1e google::LogMessageFatal::~LogMessageFatal() @ 0x7f8efecb0358 caffe::ImageDataLayer<>::DataLayerSetUp() @ 0x7f8efec1a98e caffe::BasePrefetchingDataLayer<>::LayerSetUp() @ 0x7f8efeb7cd07 caffe::Net<>::Init() @ 0x7f8efeb7f44e caffe::Net<>::Net() @ 0x7f8efed094fa caffe::Solver<>::InitTrainNet() @ 0x7f8efed0a9c5 caffe::Solver<>::Init() @ 0x7f8efed0acdf caffe::Solver<>::Solver() @ 0x7f8efed02801 caffe::Creator_SGDSolver<>() @ 0x40a288 train() @ 0x407208 main @ 0x7f8efcf6a830 __libc_start_main @ 0x407ad9 _start @ (nil) (unknown)

It is looking for an 'empty' folder within '2012-12-08', but there is no such folder.. Any idea?

[fabiocarrara]

Hi @tszngai , we used the already segmented patches for PKLot, that are in the folder PKLotSegmented under the dataset root folder. I think you are pointing to the full images instead. Try replacing

/home/py_lab/deep-parking/PKLot/PKLot/

with

/home/py_lab/deep-parking/PKLot/PKLotSegmented/

as root folder in config.py.

[tszngai]

Thank you for the reply @fabiocarrara ! I replaced it with the PKLotSegmented and after 20 hours of training, new errors came up..

I0712 06:15:46.586170 9696 solver.cpp:414] Test net output #1: loss = 0.335053 (* 1 = 0.335053 loss) I0712 06:15:46.586190 9696 solver.cpp:347] Iteration 444, Testing net (#3) I0712 06:17:01.682238 9696 blocking_queue.cpp:49] Waiting for data DEBUG 2018-07-12 06:18:04: Summarizing mAlexNet-tr_PKLot_train-vl_PKLot_val_CNRPark-EXT_val-ts_PKLot_test_CNRPark-EXT_test ... Traceback (most recent call last): File "main.py", line 104, in pyffe.summarize(exps).to_csv('results.csv') File "/home/py_lab/deep-parking/pyffe/experiment.py", line 84, in summarize r = e.summarize() File "/home/py_lab/deep-parking/pyffe/experiment.py", line 42, in decorator return function(*args, **kwargs) File "/home/py_lab/deep-parking/pyffe/experiment.py", line 548, in summarize last_iter = log_data['train']['iteration'][-1] IndexError: list index out of range

[fabiocarrara]

It seems that one of the log files of the experiments is not properly formatted. It may indicate there was a problem with a specific experiment. Please, manually check the log files in the runs/ subfolders for errors. You may then rerun only the failed experiments by commenting the finished ones in main.py.

If there are no failed experiments, there is probably a bug in the summarize() function. However, you could always process the results of the finished experiments by parsing the logs in other ways, e.g by grepping or using external code.

[tszngai]

Thanks again for your response @fabiocarrara . Now that I have my own training models, I would like to use them to write a function that can detect vehicles from any input images. Could you advice the approaches or if there are any useful tutorials you know? Thank you!