grimoirelab-elk
grimoirelab-elk copied to clipboard
[enrich] Update decorator to add metadeta only for non-empty items
The decorator present in the below method adds metadata to the enriched item even if the item is empty https://github.com/chaoss/grimoirelab-elk/blob/48486d09092f466b21572ac3b9182f4e6310c544/grimoire_elk/enriched/enrich.py#L84
An empty item may be generated in case the item does not belong to a category that is to be enriched. For e.g in case, the item does not belong to 'issues', 'pull requests' or 'repositories' in the case of github2 enricher. https://github.com/chaoss/grimoirelab-elk/blob/48486d09092f466b21572ac3b9182f4e6310c544/grimoire_elk/enriched/github2.py#L219
An update is also needed here to upload only non-empty documents: https://github.com/chaoss/grimoirelab-elk/blob/48486d09092f466b21572ac3b9182f4e6310c544/grimoire_elk/enriched/github2.py#L389
Proposed solution:
- Add the below check to https://github.com/chaoss/grimoirelab-elk/blob/48486d09092f466b21572ac3b9182f4e6310c544/grimoire_elk/enriched/enrich.py#L93
eitem = func(self, *args, **kwargs)
+ if not eitem:
+ return eitem
- Add a check to skip uploading of empty documents in the
enrich_items
method. A continue statement can be added here: https://github.com/chaoss/grimoirelab-elk/blob/578b494d38e45da5b4c100bde19718be60c7ca91/grimoire_elk/enriched/enrich.py#L390
- rich_item = self.get_rich_item(item)
+ if not rich_item:
+ continue
- Add a check the enricher for backends which override the
enrich_items
method e.g. in : https://github.com/chaoss/grimoirelab-elk/blob/48486d09092f466b21572ac3b9182f4e6310c544/grimoire_elk/enriched/github2.py#L389
eitem = self.get_rich_item(item)
- items_to_enrich.append(eitem)
+ if eitem:
+ items_to_enrich.append(eitem)
Hi, @valeriocos. Let me know if it looks good to you :)
Hi @animeshk08 , your solution looks good, however I don't know if these changes may have side effects. I understand that they shouldn't, but I'm not sure.
Okay, Let me know how we should move ahead.
I understand that they shouldn't, but I'm not sure.
I think the same.
@animeshk08 , can you prepare a pull request with the suggested changes. I'll perform with it some long-execution tests. Thanks!
Thanks. I am on it!!