Error when running `examples/py_api/api_example.py`

[leslie-fang-intel]

We find when we short the tuning steps in examples/py_api/api_example.py, for example, from default 500 to 2. We will get error in api.finish as error message:

(runtime) [root@CPX-4 py_api]# python api_example.py
[     1s]    INFO opentuner.search.plugin.DisplayPlugin: tests=2, best {'x': -127}, cost time=18769.0000, found by NormalGreedyMutation
Traceback (most recent call last):
  File "api_example.py", line 59, in <module>
    main()
  File "api_example.py", line 54, in main
    api.finish()
  File "/home/lesliefang/pytorch_1_7_1/runtime/opentuner/opentuner/api.py", line 81, in finish
    self.search_driver.best_result.configuration)
AttributeError: 'NoneType' object has no attribute 'configuration'

@jansel Could you help to take a look? BTW: we are using the latest commit of opentuner as 05e2d6b953.

[jansel]

This almost looks like it didn't run for long enough since self.search_driver.best_result is None, how many iterations is this?

[leslie-fang-intel]

Hi thanks for the reply. Yes, I think so. It‘ only 2 iterations. But since we only use the total tune time as limitation. For a large problem or on a system which running slow, we may get small tuning iterations, is there a way to avoid this error from happening?

[leslie-fang-intel]

Or in practical, at least how many iterations should we set to avoid this problem?

[jansel]

It depends on the search algorithm, many require some warmup iterations. If you set iterations to 100 or 1000 does it fix the problem?

You could make this error go away by adding an if self.search_driver.best_result is not None around the error line (and the one after), but the tuning run will still have found no result.

[leslie-fang-intel]

Yes, a longer iterations will not see this problem. So I think I need to add some try...catch... flow to handle the case when self.search_driver.best_result is None.