What is a simpler method to detect query execution failures?

[Wangyibo321]

I am utilizing Benchbase to conduct benchmark testing for knob tuning. Occasionally, certain configurations of the knobs can cause some queries to fail during execution. In such instances, it is necessary to impose a penalty on the tuning algorithm. However, I encounter difficulty in capturing exceptions when automating the tuning process with commands like os.system(). My current workaround involves redirecting Benchbase's output to a file and scanning for any queries listed after "Unexpected SQL Errors" to determine unsuccessful executions. Is there a more efficient way to accurately ascertain if the entire workload has been successfully executed and to identify which queries have failed?