Gitlab integration

[juanjovazquez]

closes #43

[codecov-io]

Codecov Report

Merging #44 into master will increase coverage by 1.56%. The diff coverage is 13.92%.

@@            Coverage Diff             @@
##           master      #44      +/-   ##
==========================================
+ Coverage   58.52%   60.08%   +1.56%     
==========================================
  Files         134      130       -4     
  Lines        4330     4224     -106     
  Branches      107      121      +14     
==========================================
+ Hits         2534     2538       +4     
+ Misses       1796     1686     -110

Impacted Files	Coverage Δ
core/src/main/scala/Github.scala	26.59% <ø> (ø)	:arrow_up:
core/src/main/scala/audit/Auditor.scala	81.81% <ø> (ø)	:arrow_up:
core/src/main/scala/audit/AuditEvent.scala	100% <ø> (ø)	:arrow_up:
core/src/main/scala/Json.scala	0% <0%> (ø)	:arrow_up:
core/src/main/scala/Gitlab.scala	0% <0%> (ø)
core/src/main/scala/Repo.scala	50% <0%> (-23.34%)	:arrow_down:
core/src/main/scala/Released.scala	100% <100%> (ø)	:arrow_up:
core/src/main/scala/storage/h2.scala	82.56% <100%> (ø)	:arrow_up:
http/src/main/scala/plans/Auth.scala	96.42% <100%> (ø)	:arrow_up:
core/src/main/scala/User.scala	100% <100%> (ø)	:arrow_up:
... and 12 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 1bf8a4d...ff26ba4. Read the comment docs.

[timperrett]

@juanjovazquez what's the status of this PR? I would like to label it with a status for admin purposes :-)

[juanjovazquez]

@timperrett It's almost all covered except releases and webhooks. This week I'll spend some time trying to get these two aspects finished but, as I mentioned in the issue, the releases part changes quite a bit with respect to Github.

My goal so far has been not to be very disruptive with respect to the current code in order to have a first version working as soon as possible. Later, algebra and interpreters could be refactored to better accommodate the three most important scms: Github, Gitlab and Bitbucket.

If you prefer, we could merge after rebasing as it is now in order to tackle what is missing in individual issues. This PR starts to be too big. Please, let me know what works better for you.

[juanjovazquez]

@timperrett webhooks client api implemented. However, I bumped into another drawback: Gitlab's hooks won't set the User-Agent header. Reported here and looking for a workaround. :angry:

[juanjovazquez]

@timperrett Definitely stuck with the release part. Basically, Gitlab links the concept of release completely to the tag. There is no way to identify the release by itself. And the worst thing is that tags don't have a numerical identifier, but basically the reference as a String, e.g. refs/tags/v1.0.0.

So, my plan is to modify the id type for ReleaseEvent and Release to String. Done this, I'd listen for tags creation events and check if this taghas release notes. In that case, I'd consider it a release.

Quite hacky, but to be honest, I can't find a better way to accommodate this requirement without tackling a profound refactoring. So, if you agree, I'll implement it this way, at least in a first version.

[timperrett]

@juanjovazquez is there the ability to attach files to a release / tag in GitLab? That's going to be really important so hope it is available (otherwise this might be moot).

@adelbertc @kaiserpelagic can you just chime in with anything you can think of that might cause a problem with releases having a string ID? Provided they never collide then it should be fine. The only drawback might be table indexing, but I dont recall off the top of my head which columns are indexed; im thinking its not a huge problem.

[juanjovazquez]

@timperrett It's a bit hacky but yes, Gitlab does offer the capability of attaching files to tags as release notes through the web interface. It is hacky because Gitlab just uploads the file generating an GUID for it and adds the url as a yaml comment to the description box. So, it's feasible to retrieve this url from the tag info as release notes. An excerpt of the tag info fetched from the API (see the release section, it's the only we have):

  {
    "commit": {
      "id": "2695effb5807a22ff3d138d593fd856244e155e7",
      "short_id": "2695effb",
      "title": "Initial commit",
      "created_at": "2017-07-26T11:08:53.000+02:00",
      "parent_ids": [
        "2a4b78934375d7f53875269ffd4f45fd83a84ebe"
      ],
      "message": "Initial commit",
      "author_name": "John Smith",
      "author_email": "[email protected]",
      "authored_date": "2012-05-28T04:42:42-07:00",
      "committer_name": "Jack Smith",
      "committer_email": "[email protected]",
      "committed_date": "2012-05-28T04:42:42-07:00"
    },
    "release": {
      "tag_name": "1.0.0",
      "description": "[foo.json](/uploads/<guid>/foo.json)"
    },
    "name": "v1.0.0",
    "message": null
  }

When there's no release, the release section in the json is null. So, at least in theory we might identify whether the creation of some tag involves a release or not.

As for the asset info. I won't be able to provide ID's or other fields like name or state. Just the url and of course the content. I might use dummy values for the rest of fields that are not available.

Anyway, I'd still need to change the ID type for releases in order to use tags with release notes as Nelson releases.

[juanjovazquez]

@timperrett I've updated the PR modifying the Id type for Release and ReleaseEvent. Please, tell me whether you're comfortable with this change so that I can finish the implementation. At first sight the change doesn't seem problematic but maybe I missed something.