open_model_zoo icon indicating copy to clipboard operation
open_model_zoo copied to clipboard

Published 20 hours ago • verified publisher icon openvinotoolkit

Correctness checking for security_barrier_camera_demo w/ 1 network multi channels with inputting 1 image

Open yangwang201911 opened this issue 3 years ago • 6 comments

Correctness checking for demo security_barrier_camera_demo with single image inputting for each channel.

JIRA: https://jira.devtools.intel.com/browse/CVS-74969

yangwang201911 avatar Mar 23 '22 03:03 yangwang201911

there are some basic pre-commit checks failed:

06:15:26 + ./venv/bin/python ./src/ci/check-basics.py 06:15:26 running miscellaneous checks... 06:15:27 demos/tests/correctness_checker.py:26: trailing whitespace. 06:15:27 +
06:15:27 demos/tests/correctness_checker.py:29: trailing whitespace. 06:15:27 + pass 06:15:27 demos/tests/correctness_checker.py:61: trailing whitespace. 06:15:27 +
06:15:27 demos/tests/correctness_checker.py:66: trailing whitespace. 06:15:27 + index += 1 06:15:27 demos/tests/correctness_checker.py:68: trailing whitespace. 06:15:27 +
06:15:27 demos/tests/correctness_checker.py:94: trailing whitespace. 06:15:27 +
06:15:27 demos/tests/correctness_checker.py:120: trailing whitespace. 06:15:27 +
06:15:27 demos/tests/run_tests.py:269: trailing whitespace. 06:15:27 + content = '' 06:15:27 demos/tests/run_tests.py:335: trailing whitespace. 06:15:27 + rawResults = '' 06:15:27 demos/tests/run_tests.py:336: trailing whitespace. 06:15:27 + execution_time = -1 06:15:27 demos/tests/run_tests.py:360: trailing whitespace. 06:15:27 +
06:15:27 running yamllint... 06:15:35 running flake8... 06:15:47 ./demos/tests/run_tests.py:36:1: F401 'operator.contains' imported but unused 06:15:47 ./demos/tests/correctness_checker.py:18:1: F401 'unittest.case' imported but unused 06:15:47 ./demos/tests/correctness_checker.py:65:20: E111 indentation is not a multiple of 4 06:15:47 ./demos/tests/correctness_checker.py:66:24: E111 indentation is not a multiple of 4 06:15:47 ./demos/tests/correctness_checker.py:67:20: E111 indentation is not a multiple of 4 06:15:47 ./demos/tests/correctness_checker.py:103:22: E211 whitespace before '(' 06:15:47 ./demos/tests/correctness_checker.py:109:22: E211 whitespace before '(' 06:15:47 ./demos/tests/correctness_checker.py:113:21: F402 import 'case' from line 18 shadowed by loop variable 06:15:47 ./demos/tests/correctness_checker.py:114:26: E211 whitespace before '(' 06:15:47 ./demos/tests/correctness_checker.py:116:30: E211 whitespace before '(' 06:15:47 ./demos/tests/correctness_checker.py:117:26: E211 whitespace before '(' 06:15:47 ./demos/tests/correctness_checker.py:126:4: E111 indentation is not a multiple of 4 06:15:47 ./demos/tests/correctness_checker.py:127:8: E111 indentation is not a multiple of 4 06:15:47 ./demos/tests/correctness_checker.py:127:13: E211 whitespace before '(' 06:15:47 ./demos/tests/correctness_checker.py:128:8: E111 indentation is not a multiple of 4 06:15:47 ./demos/tests/correctness_checker.py:129:12: E111 indentation is not a multiple of 4 06:15:47 ./demos/tests/correctness_checker.py:130:8: E111 indentation is not a multiple of 4 06:15:47 ./demos/tests/correctness_checker.py:131:12: E111 indentation is not a multiple of 4

vladimir-dudnik avatar Mar 23 '22 09:03 vladimir-dudnik

I'm generally fine with the changes, yet the correctness checking seems to happen on the OMZ side? how do we get the results of such correctness, is there a weekly/nightly etc report or what? also the question to @vladimir-dudnik

myshevts avatar Mar 23 '22 12:03 myshevts

@myshevts the idea is to extend OMZ tests with inference results correctness check (currently we do not have this). OMZ tests currently used in automatic public runs, but there is no any reports on this. Question on regular official validation runs and reports should be addressed to OpenVINO validation team. Our goal is to provide tools for that.

vladimir-dudnik avatar Mar 23 '22 21:03 vladimir-dudnik

there are still some precommit checks failed 09:06:00 running flake8... 09:06:10 ./demos/tests/run_tests.py:36:1: F401 'operator.contains' imported but unused

09:06:39 running documentation checks... 09:06:40 demos/tests/correctness_checker.py:133: last line doesn't end with a newline character

vladimir-dudnik avatar Mar 29 '22 07:03 vladimir-dudnik

@yangwang201911 it seems change in threads_common.cpp brings issue for social_distance_demo

vladimir-dudnik avatar Apr 01 '22 14:04 vladimir-dudnik

Revert the file threads_common.cpp

yangwang201911 avatar Apr 02 '22 02:04 yangwang201911