Add new option "Timeout" for pipeline jobs

[michelvocks]

By default, a job from a pipeline run is running for ever and this is the expected behavior. We don't know how long a job should run and therefore we don't set a global timeout for pipeline jobs.

Therefore, we should provide the user the possibility to manually set a job timeout via job declaration: https://github.com/gaia-pipeline/protobuf/blob/master/plugin.proto#L15

This also includes updates for all SDKs. For example: https://github.com/gaia-pipeline/gosdk/blob/master/job.go#L29

[Skarlso]

I wonder. The plugins have health checks. We could intentionally ping a plugin if it's still healthy and just working on something for a long time. If a healthcheck fails after X times of trying, I think it should be safe to kill the job off. If the health check fails that generally would mean that the plugin/job is dead. Not?

EDIT: Never mind. A job could still be frozen while the plugin itself responds as healthy.

[michelvocks]

That's already implemented by the go-plugin. If the health check does not respond, go-plugin will terminate the process. At least that is what I can read from the code 😄

[Skarlso]

I'll add this. Plus the UI for it.