feat: added retry on files sync error

[idsulik]

Description

Hi! I've been using skaffold for the last 2 years and from time to time I see error - "Skipping deploy due to sync error" and you can't do nothing with it in it was executed in auto sync mode.
Within this PR I added retry mechanism if sync returns error

[google-cla[bot]]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[idsulik]

@ericzzzzzzz Hi! Should I do anything else or I just need to wait?

[ericzzzzzzz]

Hi @idsulik I think this needs rebase, also do we know what errors are retryable and what not? I'm thinking we should distinguish those if we implement a retry mechanism for sync error

[idsulik]

@ericzzzzzzz I think we should retry sync regardless of the error because in the case of auto-sync mode, you can't force it to sync again. there are several cases:

• pre/post hooks error.
• copying/deleting docker files.

So if something goes wrong it can retry it because otherwise I have to use a workaround - checkout a different branch, checkout back to force it to sync changes

[idsulik]

do you know any error related to the sync when we shouldn't retry ?

[ericzzzzzzz]

@idsulik You can find a few errors which are not recoverable without manual work using the query in "sync error" in:title.

[idsulik]

@ericzzzzzzz what do you mean? Search for "sync error" in the project gives me only 3 lines:

I've already specified these errors above. Could you please specify 1 sync error that is not recoverable without manual action?

[ericzzzzzzz]

I mean you can use the query on this github repo, to find the related issues. Like permission error.

[ericzzzzzzz]

@ericzzzzzzz I think we should retry sync regardless of the error because in the case of auto-sync mode, you can't force it to sync again. there are several cases:

• pre/post hooks error.
• copying/deleting docker files.

So if something goes wrong it can retry it because otherwise I have to use a workaround - checkout a different branch, checkout back to force it to sync changes

• pre/post hooks error. Do you mean hooks error will cause sync fail and we should re-try sync in this case? Cloud you give me a specific example why re-try sync would help ?

• copying/deleting docker files. Wouldn't this trigger a rebuild?

[idsulik]

• Cloud you give me a specific example why re-try sync would help ?

while skaffold is trying to sync files something happened with the connection, skaffold will fail and when the connection become stable, you have to somehow trigger skaffold to start syncing again. if we have retry, on second/third retry it can sync the files successfully .

I have such cases a lot(VPN issue, quick checkout between branches lead to sync error etc.)

[ericzzzzzzz]

• Cloud you give me a specific example why re-try sync would help ?

while skaffold is trying to sync files something happened with the connection, skaffold will fail and when the connection become stable, you have to somehow trigger skaffold to start syncing again. if we have retry, on second/third retry it can sync the files successfully .

I have such cases a lot(VPN issue, quick checkout between branches lead to sync error etc.)

@idsulik thank you for the explanation! Could you please fix the ci issue. I'll approve the change.

[idsulik]

@ericzzzzzzz pushed the fix. thank you!