Can't create pipeline

[synergiator]

Description of the problem, including code/CLI snippet

Can't create a pipeline, getting error 500.

project = self.gitlab.projects.get(project_id)

pipeline = project.pipelines.create({'ref': ref, 'variables': [variables]})

Expected Behavior

Pipeline creation successful

Actual Behavior

Error 500:

gitlab.exceptions.GitlabCreateError: 500: 

500 Internal Server Error

Specifications

I can reproduce the problem in two slightly different environments with exactly same behavior.

Please consider also the additional section "Test control logic"

Envrionment 1

• Python version: 3.8.5
• python-gitlab version: 2.5.0
  • API version you are using (v3/v4): v4
  • Gitlab server version: 13.4.5-ee

Environment 2

• Python version: 3.10.0a2
• python-gitlab version: 2.5.0
  • API version you are using (v3/v4): v4
  • Gitlab server version: 13.5.4

Test control logic

Assuming there might be indeed a problem in the context where/how I use the python-gitlab lib I thought it might be helpful to run in parallel an API call using curl. Indeed the following call does create a pipeline in my target GitLab project.

curl --request POST --header 'PRIVATE-TOKEN: [MASKED]' 'GITLAB_URL//api/v4/projects/3/pipeline?ref=master' -d '[{ '\''CI_VARIABLE'\'': '\''IT WORKS!! SOURCE PIPELINE 12'\'' }]'

The pipeline is though not successful; as it seems, there is some auth problem, though the access token has all access rights you can give it for a repo.

Reinitialized existing Git repository in /builds/plab/dynamicpipelines/target/.git/
remote: HTTP Basic: Access denied
fatal: Authentication failed for 'https://GITLAB_URL/plab/dynamicpipelines/target.git/'

[janvidar]

I had a problem quite similar to this, which is partly caused by poor documentation, but also unexpected API design.

With your curl example you are essentially setting one variable: CI_VARIABLE=''IT WORKS"

It may look tempting to use the python API like this:

project.pipelines.create({'ref': 'main', 'variables': { 'CI_VARIABLE': 'IT WORKS' }})

However, reading the documentation a bit more carefully shows that you instead should send an array of objects that are formatted with explicit 'key' and 'value' keys.

Like this:

project.pipelines.create({'ref': 'main', 'variables': [{ 'key': 'CI_VARIABLE', 'value': 'IT WORKS' }]})

I am not sure if this was your problem?

[max-wittig]

Yes. If you ever see errors, such as 500 the library can't be blamed. That's always GitLab's fault, we sadly can't fix those.

[janvidar]

I understand why you would say that.

However, if it is expected that the user of the library should be intimately familiar with the inner details of the Gitlab REST API and encoding, then I am not sure what this library exists for.

[max-wittig]

@janvidar 500 means server error. It doesn't mean that you did something wrong, but that GitLab did something wrong.

Maybe your instance is broken, overloaded, etc. I don't know. Also 13.4.5-ee is really old and very vulnerable to all kind of security leaks and code execution. You should upgrade immediately.

Maybe also there was a bug in GitLab regarding this feature in this version.

[janvidar]

@max-wittig Hi, I didn't file this report, nor do I run this old version of GL.

But, this issue has been open for quite some time. I added my comments above after having a similar issue, which was caused by unclear documentation and a similarly confusing API of this library.

The fact that GL returns 500 when you get things "wrong" is of course wrong - I get that. This happens in 14.4.5 (almost current) also, unless you know how to specially format your variables section of the project.pipelines.create call. ... So, that was my contribution here.

[max-wittig]

@janvidar The thing is that we want to provide a light wrapper around the API here. GitLab should ideally return 400 with the correct error message. If we add tons of client logic and GitLab changes around everything, we need to adapt that.

With the current approach, we minimize work and keep this project maintainable. But you're welcome to contribute to it, if you have some good ideas!