grimoirelab-elk icon indicating copy to clipboard operation
grimoirelab-elk copied to clipboard

[enrich] Update decorator to add metadeta only for non-empty items

Open animeshk08 opened this issue 4 years ago • 6 comments

The decorator present in the below method adds metadata to the enriched item even if the item is empty https://github.com/chaoss/grimoirelab-elk/blob/48486d09092f466b21572ac3b9182f4e6310c544/grimoire_elk/enriched/enrich.py#L84

An empty item may be generated in case the item does not belong to a category that is to be enriched. For e.g in case, the item does not belong to 'issues', 'pull requests' or 'repositories' in the case of github2 enricher. https://github.com/chaoss/grimoirelab-elk/blob/48486d09092f466b21572ac3b9182f4e6310c544/grimoire_elk/enriched/github2.py#L219

An update is also needed here to upload only non-empty documents: https://github.com/chaoss/grimoirelab-elk/blob/48486d09092f466b21572ac3b9182f4e6310c544/grimoire_elk/enriched/github2.py#L389

animeshk08 avatar Apr 14 '20 12:04 animeshk08

Proposed solution:

  • Add the below check to https://github.com/chaoss/grimoirelab-elk/blob/48486d09092f466b21572ac3b9182f4e6310c544/grimoire_elk/enriched/enrich.py#L93
   eitem = func(self, *args, **kwargs)
 + if not eitem:
 +          return eitem
  • Add a check to skip uploading of empty documents in the enrich_items method. A continue statement can be added here: https://github.com/chaoss/grimoirelab-elk/blob/578b494d38e45da5b4c100bde19718be60c7ca91/grimoire_elk/enriched/enrich.py#L390
- rich_item = self.get_rich_item(item)
+ if not rich_item:
+       continue
  • Add a check the enricher for backends which override the enrich_items method e.g. in : https://github.com/chaoss/grimoirelab-elk/blob/48486d09092f466b21572ac3b9182f4e6310c544/grimoire_elk/enriched/github2.py#L389
  eitem = self.get_rich_item(item)
- items_to_enrich.append(eitem)
+ if eitem:
+       items_to_enrich.append(eitem)

animeshk08 avatar Apr 14 '20 12:04 animeshk08

Hi, @valeriocos. Let me know if it looks good to you :)

animeshk08 avatar Apr 14 '20 12:04 animeshk08

Hi @animeshk08 , your solution looks good, however I don't know if these changes may have side effects. I understand that they shouldn't, but I'm not sure.

valeriocos avatar Apr 15 '20 07:04 valeriocos

Okay, Let me know how we should move ahead.

I understand that they shouldn't, but I'm not sure.

I think the same.

animeshk08 avatar Apr 15 '20 07:04 animeshk08

@animeshk08 , can you prepare a pull request with the suggested changes. I'll perform with it some long-execution tests. Thanks!

valeriocos avatar Apr 15 '20 12:04 valeriocos

Thanks. I am on it!!

animeshk08 avatar Apr 15 '20 12:04 animeshk08