DD Silently accepts `test_type` mismatch

[kiblik]

Be informative DD supports the creation of custom test_type if the parser supports get_tests. The report can contain custom test_type which is useful during deduplication. test_type is set during the first import.

Bug description If a user reimports the file that contains a different test_type, DD silently accepts this mismatch. But it should not.

Steps to reproduce Steps to reproduce the behavior:

1. Import report 1 as generic
2. Import report 2 as generic
3. See that test_type is still the same

Expected behavior I suppose there are 2 possible ways (I can imagine that both would be implemented and users can choose which is better for their deployment; but I would default with fail):

• Fail: When DD detects inconsistency between the original test type and the test type coming from a new report, it should raise some error (before starting deduplication and writings to DB)
• Adopt: Update test_type. Before deduplication starts.

Open question: Right now DD supporters only one test type for report. Is this going to change in the future? How should reimport behave in that case?

Environment information

• DefectDojo version: [2024-05-16 12:17:39 -0500] f44c975c1: Bump python-gitlab from 4.4.0 to 4.5.0 (#10209) [ (HEAD, upstream/dev)]

Screenshots

Sample scan files

• generic-report-test-type-1.json
• generic-report-test-type-2.json

[Maffooch]

This is a question that has been in my mind recently. I think the largest consideration here would have to be around deduplication settings. In the case of updating the test type name from import1 to import2, there maybe be different deduplication settings between the two test types, and lead to some weird behavior. Primary reason for this is that the hash codes for findings under test type 1 would not be updated when suddenly the findings are now under test type 2.

For that reason, I believe that only leaves one option that you had proposed:

Fail: When DD detects inconsistency between the original test type and the test type coming from a new report, it should raise some error (before starting deduplication and writings to DB)

I believe this could be a simple check for when the test already exists: Does the calculated test type name from the internal test match the test type name that currently exists in the system? If not, raise an exception

[valentijnscholten]

@kiblik what's the status of this?