[Improvement]: The optimization process stuck at ACK status when the task execution encounters unexpected exceptions

[Jzjsnow]

Search before asking

• [x] I have searched in the issues and found no similar issues.

What would you like to be improved?

Currently, the ams optimization process gets stuck when the optimizer executor encounters an unexpected exception while executing a task.

The reason for this problem is that when the optimizer throws an exception during the execution of a task, it only logs “Optimizer executor[xx] got an unexpected error”, and doesn't call ams about the completion of the task. The task on ams is stuck in the ACK status and the optimization process is stuck at MINOR/MAJOR/FULL.

How should we improve?

• When the optimizer throws an exception during the execution of a task, notify ams the completion of the task.
• Detect the task execution time on ams server, support timeout retry.

Are you willing to submit PR?

• [x] Yes I am willing to submit a PR!

Subtasks

No response

Code of Conduct

• [x] I agree to follow this project's Code of Conduct