[gitlabcomments] Add enricher to handle gitlab comments

[vchrombie]

This PR proposes a new gitlab enricher to handle gitlab comments. The old enricher is left in place to avoid breaking changes with existing customer dashboards.

Ref: https://github.com/chaoss/grimoirelab/issues/208

[vchrombie]

The PR has a lot of pending work, I will be working on this. I will mark ready for review after completing some good work.

Please let me know if you are having any suggestions. Thanks.

[valeriocos]

Hi @vchrombie , please ping me when I can review this PR. I had a look at the current status and the start looks promising :)

WRT tests (if this can be of any help), you should run Perceval and copy some docs to a test file, which will be finally put in the folder tests/data. The name is important since it is used to automatically identify the file and load its content (https://github.com/chaoss/grimoirelab-elk/blob/master/tests/base.py#L137)

If needed you can tweak the test data (e.g., changing value attributes) to make sure that all code is covered when running the tests.

[vchrombie]

Hi @valeriocos I have completed the major part of the PR. I am yet to complete the __get_reactions function and I am working on that right now.

[valeriocos]

Thanks for the update @vchrombie

[vchrombie]

I will mark this PR ready-for-review once I complete the __get_reactions function too. :rocket:

[coveralls]

Pull Request Test Coverage Report for Build 1227039055

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 30 unchanged lines in 1 file lost coverage.
• Overall coverage increased (+0.5%) to 82.719%

---

Files with Coverage Reduction	New Missed Lines	%
/home/runner/work/grimoirelab-elk/grimoirelab-elk/grimoire_elk/utils.py	30	66.54%
<!--	Total:	30

Totals
Change from base Build 1220934812:	0.5%
Covered Lines:	9042
Relevant Lines:	10931

---

💛 - Coveralls

[vchrombie]

https://github.com/chaoss/grimoirelab-elk/pull/881#pullrequestreview-425802955

Thanks for the review @valeriocos. I have just updated the PR with the changes you have suggested.

Can you add the tests and schema?

Sure, I will be working on it and add them here soon.

[vchrombie]

Hi @valeriocos

Recent updates to the PR

1. I removed the reactions mapping as we discussed it is not needed and also implemented the new __get-reactions() function.
2. added the schema using the gitlabcomments-mrs/issues_enriched index.
3. added the tests, (highly need a review on this as I'm not really good with it).

I'm right now checking the coverage, will update the PR if needed.

Once the PR is perfect, I can squash the commits and rebase them with the latest changes too.

[vchrombie]

Thanks @valeriocos. I left a lot of things pending, my bad. I will get back to you after fixing them. :+1:

[valeriocos]

No worries, thank you for working on this PR

[vchrombie]

Also, I have made a small script to generate schema file of an es index, generate-es-index-schema-py. This was built on top of @animeshk08's script as it had some backlogs.

The script can be improved by adding the idea of default field and adding descriptions. Please leave your comments in the gist or you can open an issue here, vchrombie/grimoirelab-scripts. Feedback is highly welcomed. ^^

[vchrombie]

Hi @valeriocos I have updated the PR as per your suggestions. Can you review this when you are free? Thanks.

[valeriocos]

please @vchrombie ping me when the PR is ready for review, thanks

[almereyda]

Coming from outside and having dismissed GrimoireLab previously for lacking GitLab support, I would still like to see it support additional code forges other than GitHub, as per the arguments in inhttps://github.com/chaoss/grimoirelab/issues/284#issuecomment-827155190).

Is there something me as an outsider can contribute to the formal aspects of the PR, in so the remaining technical questions can be answered more easily? Speaking of:

• [ ] rebase on master
• [ ] update elk readme
• [ ] add mordred supported data sources documentation (PR)
• [ ] update tests for edge cases and error conditions

[vchrombie]

Hi @almereyda, thanks for your comment.

Coming from outside and having dismissed GrimoireLab previously for lacking GitLab support

Sorry about it, but the current GrimoireLab supports analyzing issues and merge requests from the gitlab projects (reference). This PR is an additional enricher which can analyze the comments of the gitlab issues and mrs. I couldn't complete it because I completely forgot about this.

Is there something me as an outsider can contribute to the formal aspects of the PR, in so the remaining technical questions can be answered more easily? Speaking of:

Thanks for offering help, I appreciate it. But, I think I can complete the work on this PR. I can pull some time during the coming weekend and complete the pending work.