Agent: The attempt to validate pktvisor readiness is very fast and causes the agent to fail in less than 1 second

[manrodrigues]

From the first attempt to the last attempt there was an interval of 7 milliseconds.. after that the agent was refused connection.

The time between retries could be longer to avoid this type of error

{"level":"info","ts":"2022-07-27T16:56:18.712Z","caller":"pktvisor/pktvisor.go:294","msg":"pktvisor process started","pid":20}

{"level":"info","ts":"2022-07-27T16:56:18.713Z","caller":"pktvisor/pktvisor.go:304","msg":"pktvisor is not ready, trying again"}

{"level":"info","ts":"2022-07-27T16:56:18.714Z","caller":"pktvisor/pktvisor.go:304","msg":"pktvisor is not ready, trying again"}

{"level":"info","ts":"2022-07-27T16:56:18.714Z","caller":"pktvisor/pktvisor.go:304","msg":"pktvisor is not ready, trying again"}

{"level":"info","ts":"2022-07-27T16:56:18.715Z","caller":"pktvisor/pktvisor.go:304","msg":"pktvisor is not ready, trying again"}

{"level":"info","ts":"2022-07-27T16:56:18.715Z","caller":"pktvisor/pktvisor.go:304","msg":"pktvisor is not ready, trying again"}

{"level":"info","ts":"2022-07-27T16:56:18.716Z","caller":"pktvisor/pktvisor.go:304","msg":"pktvisor is not ready, trying again"}

{"level":"info","ts":"2022-07-27T16:56:18.716Z","caller":"pktvisor/pktvisor.go:304","msg":"pktvisor is not ready, trying again"}

{"level":"info","ts":"2022-07-27T16:56:18.717Z","caller":"pktvisor/pktvisor.go:304","msg":"pktvisor is not ready, trying again"}

{"level":"info","ts":"2022-07-27T16:56:18.718Z","caller":"pktvisor/pktvisor.go:304","msg":"pktvisor is not ready, trying again"}

{"level":"info","ts":"2022-07-27T16:56:18.719Z","caller":"pktvisor/pktvisor.go:304","msg":"pktvisor is not ready, trying again"}

{"level":"error","ts":"2022-07-27T16:56:18.719Z","caller":"pktvisor/pktvisor.go:308","msg":"pktvisor error on readiness","error":"Get \"http://localhost:40701/api/v1/metrics/app\": dial tcp 127.0.0.1:40701: connect: connection refused"}

{"level":"error","ts":"2022-07-27T16:56:18.719Z","caller":"agent/main.go:109","msg":"agent startup error","error":"Get \"http://localhost:40701/api/v1/metrics/app\": dial tcp 127.0.0.1:40701: connect: connection refused"}

Full logs:

https://app.zenhub.com/files/340174841/ee3c2208-9f37-4259-b474-265e72dd5d09/download

[manrodrigues]

_orb-agent-int-testBiooH_logs.txt

[lpegoraro]

PR https://github.com/ns1labs/orb/pull/1663

[lpegoraro]

This was fixed it has now a backoff

{"level":"info","ts":"2022-08-17T12:41:01.584-0300","caller":"pktvisor/pktvisor.go:267","msg":"pktvisor stdout","log":"Waiting 8 to start pktvisor"} {"level":"info","ts":"2022-08-17T12:41:02.583-0300","caller":"pktvisor/pktvisor.go:296","msg":"pktvisor process started","pid":677044} {"level":"info","ts":"2022-08-17T12:41:02.585-0300","caller":"pktvisor/pktvisor.go:307","msg":"pktvisor is not ready, trying again with backoff","backoff backoffDuration":"0s"} {"level":"info","ts":"2022-08-17T12:41:02.586-0300","caller":"pktvisor/pktvisor.go:307","msg":"pktvisor is not ready, trying again with backoff","backoff backoffDuration":"1s"} {"level":"info","ts":"2022-08-17T12:41:03.587-0300","caller":"pktvisor/pktvisor.go:307","msg":"pktvisor is not ready, trying again with backoff","backoff backoffDuration":"2s"} {"level":"info","ts":"2022-08-17T12:41:05.589-0300","caller":"pktvisor/pktvisor.go:307","msg":"pktvisor is not ready, trying again with backoff","backoff backoffDuration":"3s"} {"level":"info","ts":"2022-08-17T12:41:05.686-0300","caller":"pktvisor/pktvisor.go:267","msg":"pktvisor stdout","log":"[2022-08-17 12:41:05.686] [visor] [info] pktvisor 4.0.0 starting up"}

[manrodrigues]

ok