grimoirelab
grimoirelab copied to clipboard
chaoss
[feature request] can we still have p2o.py maintained in grimoire stack?
Hi, it was suggested by @valeriocos that I can create a feature request:
Imagine the situation that you have a full system build around the grimoire stack and orchestrating individual endpoints enrichment using the simplest possible tool.
We need something to get data from data sources (obviously this could be a Perceval) but then we need to enrich this data using SortingHat DB and put into the elastic search. Currently only p2o.py from the ELK can do this.
We don't want a full solution like (micro-)mordred - we just need something that is as lightweight as possible - we don't even need a Redis queue (so don't need arthur).
We have a lot of extra requirements, we maintain indexes, aliases, copying indexes, auto-merging, hiding sensitive data, detecting affiliations ranges, support per-project affiliations, affiliations roles like contributors/maintainers and much more - we have APIs on top of SH DB, we just want to orchestrate simple perceval+sortinghat+elk commands in multiple deployments (Helm, Kubernetes, Fargate, Docker, bare metal and possibly more).
p2o.py does it but seems to be left unmaintained and we are constantly requiring to apply more & more patches to keep it usable, and on the other side, we don't want to fork it.
We upstream most of the changes that can be used by chaoss but also maintain a lot of changes needed to support extra DB fields on Sorting hat and to handle many more algorithms (see this and that) - some of those changes are just too specific to our use case that they cannot be integrated into chaoss. All those tools are open-source and can be used to anyone.
We keep our patches as simple as possible and always apply them on top of the newest chaoss code.
Is there a way that you guys can keep p2o.py maintained? Asking because recently we just discovered that some function definitions were updated across chaoss/grimoire code but their invocations in p2o.py code weren't changed, which caused some problems.
