Add support for GitLab [$505]

[danirabbit]

We should:

• [ ] Put together a list of all the GitHub APIs we rely on
• [ ] Make sure those APIs have equivalents on GitLab
• [ ] Support publishing GitLab projects

---

There is a bounty on this issue on BountySource

---

There is a $505 open bounty on this issue. Add to the bounty at Bountysource.

[btkostner]

Currently, the only GitHub code in V2 is wrapped up in a service file. There are three interfaces that GitLab will need to implement:

• [ ] A code repo
• [ ] package repo
• [ ] log repo

https://github.com/elementary/houston/blob/v2-publishing/src/lib/service/type.ts

The other two things that will need to be added would include:

• [ ] database models (not done in V2)
• [ ] web hook controllers (also not done in V2).

[danirabbit]

@btkostner Can we make sure we get tickets opened up on GitLab (and linked back here with convenient checkboxes) for any missing APIs? :)

[matteeyah]

There's lots of GitLab functionality exposed via API - See https://docs.gitlab.com/ee/api/

If there's something missing and there isn't an issue about it already you can open an issue in http://gitlab.com/gitlab-org/gitlab-ce/issues

Let me know if there's anything that GitLab can do to make this easier for you.

[rupertdev]

Would you like to keep support for GitHub or transition entirely?

I'm looking at this.

[btkostner]

We will want to add this next to GitHub and keep them both working.

[rupertdev]

I assumed, just wanted to be sure, there are some extreme opinions out there :)

[rupertdev]

I have almost all of this done, is there a gitlab repo I can use in the e2e tests?

[btkostner]

elementary currently has no gitlab repos. Your best bet would be to create a temp repo on gitlab for testing.

[robinlahtinen]

Hope elementary stays on Github.

[danirabbit]

@robinkooli There are no plans for elementary to move any of our code. This issue is about adding GitLab support to AppCenter Dashboard so that 3rd party developers can publish repos hosted on both GitHub and GitLab

[rupertdev]

Does this need to be implemented in the v2 branch, master, or both?

[rupertdev]

@matteeyah and others:

sad news, looks like GitLab doesnt support release file uploads through the API.

There's been an issue open for some time: https://gitlab.com/gitlab-org/gitlab-ce/issues/31221

What path should we take here?

[danirabbit]

@rupertdev

Just V2, I think. @btkostner would be able to answer more definitively.

By release file uploads, this would mean attaching the completed deb file to the release tag right? If so, I think we can consider the release file uploads optional. This is more of a convenience and isn’t necessary for publishing in AppCenter

[matteeyah]

@rupertdev I pinged our VP of Product and relevant Product Managers in the issues. Thanks for bringing this up.

[rupertdev]

Still have a PR for this, looks like bounty has increased, would love to finish it up :)

[rupertdev]

@danrabbit what's the deal with this task?

[btkostner]

@rupertdev I've been doing a lot of non houston related things lately, like weblate and infrastructure tasks. I've also been prioritizing fixing issues with the testing / releasing process over adding new features.

Speaking of GitLab. I realized that in most of the code I wrote, I made a huge assumption that each service would have a single API endpoint. This mostly works for GitHub, but with GitLab (where it's much more feasible to host it yourself), this is not always the case. I've been thinking in the back of my head of how I want to deal with this and what would need to be changed.

[matteeyah]

@btkostner You could implement support for GitLab.com first, then add support for self-hosted instances in a later iteration. I think the bulk of people wanting to add their apps to the AppCenter will have their code hosted on .com.

[btkostner]

Whoops. Did not mean to close

[thecyberd3m0n]

I'm trying to find out how to test solution As I understood: Now: houston build works only with Github repositories After: it should work also with Gitlab repos

Which repos I can test this way? I'm trying to build houston repo with houston. First I tried houston build [email protected]:elementary/houston.git 6.2.0, next houston build [email protected]:elementary/houston.git master / v6.20 / refs/tags/v6.2.0. Result is "no reference found for shorthands"

Which repos I've could test by houston and how?

[btkostner]

@thecyberd3m0n You can't build houston with houston. Houston does not produce a deb file.

[thecyberd3m0n]

please give me example repo where I could start. I'm also on your Gitter

[btkostner]

To give a status update on this:

There is currently a PR to add support for GitLab to the worker. This only makes it possible for most of the "houston ci" stuff to work.

elementary is currently working on making Flatpak a first class citizen with houston, but how flatpaks are built compared to debs is quite different. Therefore, there will probably be significant changes made to support this. I have not started work on this.

I am currently working on updating the server side logic to make it easier to support other services besides GitHub. It's still in its early stage, but if anyone would like to help out, here is the repo: https://github.com/elementary/houston-server.

[thecyberd3m0n]

ok I'm working on it

[thecyberd3m0n]

tcd@tcd-MS-7A40:~/hdd/bountysource/houston$ houston build [email protected]:thecyberd3m0n/palette.git 2.2.0
setting url [email protected]:thecyberd3m0n/palette.git
httpPath https://gitlab.com/thecyberd3m0n/palette
Running build for [email protected]:thecyberd3m0n/palette.git version 2.2.0
cloning /tmp/houston/c3c24431-3674-4b09-9069-458f50c4719b/repository/0
Error while running build for [email protected]:thecyberd3m0n/palette.git for 2.2.0

================================================================================

# Incorrect values in Debian control file

AppCenter checked your Debian control files and found some errors

- Source should be "com.gitlab.thecyberd3m0n.palette" not Source: com.github.cassidyjames.palette
Section: x11
Maintainer: Cassidy James Blaede <[email protected]>
Build-Depends: debhelper (>= 9), libgee-0.8-dev, libgtk-3-dev, meson, valac
Standards-Version: 3.9.6
Homepage: https://github.com/cassidyjames/palette

Package: com.github.cassidyjames.palette
Architecture: any
Priority: optional
Depends: ${misc:Depends}, ${shlibs:Depends}
Description: View and copy colors from the elementary brand palette
 Access the elementary color palette on your local device

- Package should  "com.gitlab.thecyberd3m0n.palette" not Source: com.github.cassidyjames.palette
Section: x11
Maintainer: Cassidy James Blaede <[email protected]>
Build-Depends: debhelper (>= 9), libgee-0.8-dev, libgtk-3-dev, meson, valac
Standards-Version: 3.9.6
Homepage: https://github.com/cassidyjames/palette

Package: com.github.cassidyjames.palette
Architecture: any
Priority: optional
Depends: ${misc:Depends}, ${shlibs:Depends}
Description: View and copy colors from the elementary brand palette
 Access the elementary color palette on your local device

### Build Information
Affects Architecture: amd64
Affects Distribution: juno
Built with the following references:
- refs/tags/2.2.0

================================================================================

First (almost) successfull build from Gitlab. You can see additional logs about package that was built (it was Palette imported to Gitlab specially for that purpose). Errors are result of package descriptors missmatch (I should change them in Palette repo I suppose)

Additional questions:

• do you consider private repos?
• any additional things or processes that I should test out or implement? How to test them?

Clear Definition of Done will make CR quicker :)

[rupertdev]

I hate to complain, but I'm feeling a little upset about how this bounty/Issue was handled. I received some pretty active feedback at the time of my development, I delivered a PR that covered all of the acceptance criteria. When the PR was opened, I was told that it would be some time before it was reviewed. That was over a year ago.

It seems silly to let my work go to waste. I don't want to take away from what @thecyberd3m0n has done, I hope the same thing doesn't happen to them. I'm not really worried about what's going to happen when the bounty pays out because I'm not sure it ever will.

[thecyberd3m0n]

I'm planning to deliver it on monday. I don't want my work to be useless. @btkostner should I continue work? Could you review @rupertdev pr, maybe this is enough?

[btkostner]

@thecyberd3m0n I would stop that branch. Sorry I did not say that earlier. I figured you were trying out @rupertdev's branch.

@rupertdev's branch covers half of the work needed to support GitLab. It does everything needed to run houston build on a GitLab project, but does not contain the API server logic needed for things like oauth login and listing all of a person's repos.

The server component of this makes huge assumptions with GitHub, and is currently being rewritten for better support, easier testing, and overall betterness. Therefore it's pointless to try to add that logic now. You can see it's progress on it's GitHub repo.

When @rupertdev's PR is merged, it will be awarded half of this bounty.

[thecyberd3m0n]

actually I've just copied github.ts to gitlab.ts, changed some variables and added some ifs and produced log by my own, later I realized that somebody did it before me (my mistake)

Ok so if it was done, and second half requires additional integrations I'll stop my work on this task until any news.