openvino icon indicating copy to clipboard operation
openvino copied to clipboard

Added BenchmarkLayerTest class for simplifing of benchmarking

Open lohika-denis-kotov opened this issue 2 years ago • 1 comments

Details:

This PR adds BenchmarkLayerTest that simplifies benchmarking of single operator. The BenchmarkLayerTest has the following advantages:

  1. Reusing single layer test cases
  2. Almost no need code for benchmarking single operator
  3. Could be add in CI for verifying on performance regression (using Validate method)

Here is the example of using BenchmarkLayerTest:

...

struct GenerateProposalsBenchmarkTest : ov::test::BenchmarkLayerTest<GenerateProposalsLayerTest> {};

// By default benchmark tests could be disabled
TEST_P(GenerateProposalsBenchmarkTest, DISABLED_GenerateProposals_Benchmark) {
    SKIP_IF_CURRENT_TEST_IS_DISABLED()
    run("GenerateProposals", std::chrono::milliseconds(2000), 10);
}

INSTANTIATE_TEST_SUITE_P(
    smoke_GenerateProposalsBenchmarkTest,
    GenerateProposalsBenchmarkTest,
    ::testing::Combine(
        ::testing::ValuesIn(dynamicInputShape),
        ::testing::ValuesIn(min_size),
        ::testing::ValuesIn(nms_threshold),
        ::testing::ValuesIn(post_nms_count),
        ::testing::ValuesIn(pre_nms_count),
        ::testing::ValuesIn(inputTensors),
        ::testing::Values(ov::element::Type_t::f32),
        ::testing::Values(ov::element::Type_t::i32),
        ::testing::Values(CommonTestUtils::DEVICE_CPU)),
    GenerateProposalsBenchmarkTest::getTestCaseName);

...

lohika-denis-kotov avatar Aug 14 '22 16:08 lohika-denis-kotov

@iefode @maxnick @EgorDuplensky FYI

dmitry-gorokhov avatar Aug 15 '22 05:08 dmitry-gorokhov

@ceciliapeng2011 CI is passed, but PR still needs to be rebased. Maybe it is better to enable automerge for this PR ? Once CI is passed PR will be automatically merged

lohika-denis-kotov avatar Sep 29 '22 05:09 lohika-denis-kotov

@iefode Please, take a look at review again

lohika-denis-kotov avatar Oct 24 '22 13:10 lohika-denis-kotov

@iefode I've addressed comments from your latest review. Please, take a look at review again.

okurache-capgemini avatar Nov 22 '22 08:11 okurache-capgemini