tvm icon indicating copy to clipboard operation
tvm copied to clipboard

[Bug] Meta-scheduler tests failing in `test_meta_schedule_integration.py`

Open leandron opened this issue 2 years ago • 1 comments

There are 2 meta-scheduler tests failing when we run these tests in CI on machines that have torch installed. The last change I could track that touches on that part is in the context of #12520 - I didn't bisect that change.

As these tests do not usually run as part of CI - as reported in #12529 - it is not unexpected that they will only appear in the context of enabling Integration tests in AArch64. As my knowledge of meta-scheduler is limited, I'm opening this issue before marking these tests as xfails for further investigation.

Test impacted:

  • unittest.test_meta_schedule_integration.test_meta_schedule_integration_extract_from_resnet_with_filter_func
  • unittest.test_meta_schedule_integration.test_meta_schedule_integration_extract_from_resnet

test_meta_schedule_integration_extract_from_resnet_with_filter_func fails with:

AssertionError: fused_nn_contrib_conv2d_NCHWc_add
assert 'fused_nn_contrib_conv2d_NCHWc_add' in ['fused_nn_max_pool2d', 'fused_nn_adaptive_avg_pool2d', 'fused_nn_dense_add', 'fused_nn_conv2d_add', 'fused_nn_conv2d_add_1', 'fused_nn_conv2d_add_2', ...]
 +  where 'fused_nn_contrib_conv2d_NCHWc_add' = meta_schedule.ExtractedTask(0x1af313d0).task_name

test_meta_schedule_integration_extract_from_resnet fails with:

AssertionError: assert 30 == 20
 +  where 30 = len([meta_schedule.ExtractedTask(0x40ee8210), meta_schedule.ExtractedTask(0x40e0c040), meta_schedule.ExtractedTask(0x41ddab40), meta_schedule.ExtractedTask(0x42067aa0), meta_schedule.ExtractedTask(0x420990c0), meta_schedule.ExtractedTask(0x41e0cfe0), ...])
 +  and   20 = len(['fused_nn_max_pool2d', 'fused_nn_adaptive_avg_pool2d', 'fused_nn_dense_add', 'fused_nn_conv2d_add', 'fused_nn_conv2d_add_1', 'fused_nn_conv2d_add_2', ...])

Jenkins log: https://ci.tlcpack.ai/job/tvm/job/PR-11995/57/#showFailuresLink

cc @junrushao @driazati @areusch @masahi

leandron avatar Sep 08 '22 09:09 leandron

Also cc @Hzfengsy as suggested by tvm-bot in the PR.

leandron avatar Sep 08 '22 13:09 leandron

I just ran the tests after refactoring change in https://github.com/apache/tvm/pull/12895, it seems they have been fixed now.

masahi avatar Oct 07 '22 22:10 masahi

Yes, I got it fixed. Thank you for spotting this!

junrushao avatar Oct 08 '22 04:10 junrushao