ONE
ONE copied to clipboard
[CI] lightweight one-cmds test
What
Let's introduce lightweight one-cmds test.
Why
Current one-cmds
test takes sooo long time that we are not supporting one-cmds
test on CI. But, one-cmds
related errors happened not rarely. FYI, https://github.com/Samsung/ONE/issues/5514 addresses this issue. But before resolving that issue, we could have another option.
The option is to introduce lightweight one-cmds
test. "lightweight" here means that the test can be done in a feasible time with current frequency of posting PR. one-cmds
test has lots of duplicate flow in each test.
For example, onecc_001.test
and onecc_002.test
have the same flow that imports an inception model.
https://github.com/Samsung/ONE/blob/811989cc8f5aeecd7ebad42648d6733401f5e940/compiler/one-cmds/tests/onecc_001.cfg#L10-L16
https://github.com/Samsung/ONE/blob/811989cc8f5aeecd7ebad42648d6733401f5e940/compiler/one-cmds/tests/onecc_002.cfg#L10-L16
In lightweight test, we could just run the basic test that doesn't have duplicate logic.
Related: #9765
Better to use a more specific title? For example, "Remove duplicate logics in one-cmds tests"
@jinevening oh, I didn't mean to revisit current test. This test will be newly introduced for CI purpose.
Ah, I get it. You're making a new test set (maybe sampled from existing test cases). Thanks for explanation.
To run the test lightly, I'll choose a few tests among the package test. Tests will be chosen in the direction of maximizing the coverage.
In our CI, compiler build spends like 44 minutes and package test spends like 21 minutes. If I introduce a new job for the lightweight package test, it forces us to create the package first, which would need other 44 minutes again.
Instead, adding install command into existing release build job will do the trick.
Now, current nncc-release job on CI runs following one-cmds tests.
## one-import-tf -> one-optimize -> one-quantize -> one-codegen
./nncc docker-run /bin/bash -c 'cd ${NNCC_INSTALL_PATH}/test;bash onecc_006.test'
## one-import-tflite
./nncc docker-run /bin/bash -c 'cd ${NNCC_INSTALL_PATH}/test;bash onecc_014.test'
## one-import-bcq
./nncc docker-run /bin/bash -c 'cd ${NNCC_INSTALL_PATH}/test;bash onecc_015.test'
## one-import-onnx
./nncc docker-run /bin/bash -c 'cd ${NNCC_INSTALL_PATH}/test;bash onecc_016.test'
## one-pack
./nncc docker-run /bin/bash -c 'cd ${NNCC_INSTALL_PATH}/test;bash onecc_019.test'
## one-profile
./nncc docker-run /bin/bash -c 'cd ${NNCC_INSTALL_PATH}/test;bash onecc_021.test'
## one-quantize
./nncc docker-run /bin/bash -c 'cd ${NNCC_INSTALL_PATH}/test;bash onecc_023.test'
./nncc docker-run /bin/bash -c 'cd ${NNCC_INSTALL_PATH}/test;bash onecc_026.test'
## one-infer
./nncc docker-run /bin/bash -c 'cd ${NNCC_INSTALL_PATH}/test;bash onecc_027.test'
## onecc workflows
./nncc docker-run /bin/bash -c 'cd ${NNCC_INSTALL_PATH}/test;bash onecc_032.test'
./nncc docker-run /bin/bash -c 'cd ${NNCC_INSTALL_PATH}/test;bash onecc_041.test'
If you want to add other tests from one-cmds, feel free to comment them at this issue.