cds
cds copied to clipboard
ovh
gitlab sync status failed when specifying tag in the payload
Hey,
Trying to investigate to help, I added the trace level logs to CDS (I can do a pull request for this also if you are interested about this feature)
2019-05-26 08:01:43 [TRACE] vcs.handler.getCommitStatusHandler> commit_id:0.1.0 2019-05-26 08:01:43 [TRACE] vcs.gitlab.ListStatuses> doing 2019-05-26 08:01:43 [WARN] pprofLabel>recoverWrap>Handle>getCommitStatusHandler>ListStatuses: internal server error (caused by: Unable to get commit 0.1.0 statuses on gregoiremahe/REPONAME: unable to get commit statuses hash:0.1.0: GET https://gitlab.com/api/v4/projects/gregoiremahe/REPONAME/repository/commits/0.1.0/statuses: 404 {error: 404 Not Found}) error_uuid=8065f038-7f8c-11e9-bbc1-d66c36fcf179 method=GET request_uri=/vcs/gitlab/repos/gregoiremahe/REPONAME/commits/0.1.0/statuses stack_trace=GET https://gitlab.com/api/v4/projects/gregoiremahe/REPONAME/repository/commits/0.1.0/statuses: 404 {error: 404 Not Found} unable to get commit statuses hash:0.1.0 github.com/ovh/cds/sdk.WrapError /go/src/github.com/ovh/cds/sdk/error.go:733 github.com/ovh/cds/engine/vcs/gitlab.(*gitlabClient).ListStatuses /go/src/github.com/ovh/cds/engine/vcs/gitlab/client_status.go:139 github.com/ovh/cds/engine/vcs.(*Service).getCommitStatusHandler.func1 /go/src/github.com/ovh/cds/engine/vcs/vcs_handlers.go:551 github.com/ovh/cds/engine/api.(*Router).Handle.func1 /go/src/github.com/ovh/cds/engine/api/router.go:245 net/http.HandlerFunc.ServeHTTP /usr/local/go/src/net/http/server.go:1995 github.com/ovh/cds/engine/api.(*Router).recoverWrap.func1 /go/src/github.com/ovh/cds/engine/api/router.go:152 net/http.HandlerFunc.ServeHTTP /usr/local/go/src/net/http/server.go:1995 github.com/ovh/cds/vendor/github.com/gorilla/handlers.CompressHandlerLevel.func1 /go/src/github.com/ovh/cds/vendor/github.com/gorilla/handlers/compress.go:143 net/http.HandlerFunc.ServeHTTP /usr/local/go/src/net/http/server.go:1995 github.com/ovh/cds/engine/api.(*Router).pprofLabel.func1 /go/src/github.com/ovh/cds/engine/api/router.go:89 net/http.HandlerFunc.ServeHTTP /usr/local/go/src/net/http/server.go:1995 github.com/ovh/cds/vendor/github.com/gorilla/mux.(*Router).ServeHTTP /go/src/github.com/ovh/cds/vendor/github.com/gorilla/mux/mux.go:114 net/http.serverHandler.ServeHTTP /usr/local/go/src/net/http/server.go:2774 net/http.(*conn).serve /usr/local/go/src/net/http/server.go:1878 runtime.goexit /usr/local/go/src/runtime/asm_amd64.s:1337 Unable to get commit 0.1.0 statuses on gregoiremahe/REPONAME status=500 2019-05-26 08:01:43 [DEBUG] GET | 116.33711ms | /vcs/gitlab/repos/gregoiremahe/REPONAME/commits/0.1.0/statuses
It seems that the vcs_handlers wants to retrieve a commit called 0.1.0, but this is not a commit hash, so it fails.
Ok, so as far as I understand, there is 2 issues :
-
The fact that CDS failed to retrieve a commit status because it takes the tag instead of the hash for gitlab --> This causes we are unable to resync VCS status on the workflow Logs when we are trying to resync VCS status via workflow On the UI
-
The fact there is a duplicated commit status when we specify a git tag in payload Duplicated commit status logs
--- EDIT --- When we specify a git tag, CDS seems to call functions twice... Duplicated requests
Umm sometimes one over the two [DEBUG] POST | 415.122736ms | /vcs/gitlab/status calls returns an error. When this happens, there is no duplicated status on gitlab.
Sometimes, both the two POST /vcs/gitlab/status returns an error. When this happens, there is no status on gitlab at all
And when there is no error on the both calls, there is a duplicated status
Here are the errors :
--- EDIT --- About the first issue : I didn't investigate yet.
About the second issue : I did a work around to fix the problem. But I don't think this is the right way to fix it, it's more a workaround than a fix ;) Here is exacty the problem : The vcs gitlab client_status.go file seems to be called twice (at exactly the same time) when we are deploying + switching to running state. This causes the GitLab API to have two running pipeline instead of one. Finaly, when the workflow is over (success of fail), CDS calls gitlab API for his last state. --> So the second one is still seen as running by gitlab.
To workaround this, I setup a global variable (statusCache) to memorize all states given to gitlab by CDS, for avoid sending twice the same. But as the file is called twice exactly at the same time, both statusCache are seen as empty first. I added a time.Sleep() to shift the runtime. That's working, but I think we have to find why this file is called twice.
