Add purge feature

[ptrovatelli]

Is your feature request related to a problem? Please describe When not using the re-import feature, the number of findings will be very big eventually. We need a purge system but it's not trivial when using the deduplication. This issue is open to any suggestions / discussions. How are you currently handling this?

Describe the solution you'd like Core features

• Configurable in the system settings
  • activated or not (default not activated)
  • retention in days
• Only purge findings in ci-cd engagements
  • this is a first/easy approach to make sure we don't purge a finding that is still accurate (for example created during a manual security audit). See bonus part for a possible improvement
• Don't purge originals
  • Findings that have duplicates shouldn't be purged even if they are older than the retention to avoid leaving broken duplicate links. We could use the feature introduced in https://github.com/DefectDojo/django-DefectDojo/pull/4142 in order to elect a new original but I'm not in favor of it for automatic purge because the information of "when was this vulnerability first seen?" is very important and shouldn't be tampered with. In contractual situations, this information may allow answering who's responsible and/or who should pay for the fix.
  • Note that when a finding stops appearing in the security reports (because it was fixed), the original will eventually be purged when all the duplicates are purged.

Bonus features

• GUI for incomplete engagements: Purging only duplicates will leave some engagements which don't reflect the reality: some findings will be there (originals), some others will be missing. This is not a big problem because if we have set a retention of 1 year, we should know that information before 1 year is not accurate. Still we could work on something in the GUI to display the fact that an engagement is incomplete (grey background or something). When the purge starts working on an engagement, it should flag it as incomplete (meaning it's "possibly incomplete"; I don't think we need to go as far as to make sure that we did remove findings before flagging the engagement). Another (easier) approach would be to toogle this appearance automatically depending on the current retention, but in case of configuration change, it could be incorrect
• Extend the purge to non ci-cd engagement by checking if the finding is from a security report or was created manually. This could be configurable too (purge only findings in ci-cd engagement, or purge all findings that are not manually created). We could also purge closed findings

Describe alternatives you've considered use re-import

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ptrovatelli]

still required :)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ptrovatelli]

still needed

[Gby56]

@ptrovatelli I also suggested a simple trigger to launch the deduplication in a product, is that basically the idea ?

[ptrovatelli]

hello @Gby56 . This issue is about removing findings from the database. What you suggest could be done via an evolution of https://github.com/DefectDojo/django-DefectDojo/blob/master/dojo/management/commands/dedupe.py It is currently possible to run deduplication for all findings or for specific parsers but not for specific products