codeql-learninglab-actions icon indicating copy to clipboard operation
codeql-learninglab-actions copied to clipboard

Mismatch between suggested and actual alert counts

Open adityasharad opened this issue 5 years ago • 0 comments

The CSVs of expected results seems to include all raw results of the query (including those not present within the database source archive root). In the VS Code IDE, these correspond to the #select table in the results view. However, since the suggested template query has the problem metadata, the user is likely to see results in the alerts view, which filters out results in files outside the source archive root.

This leads to a mismatch in numbers, e.g. step 09 of the segv CTF has 70 results in the alerts view and 98 results in the #select view, and if you submit an incorrect solution it tells you it expects 98.

This suggests we need to either:

  • use only the interpreted alerts within the source archive in the expected results files, or
  • better explain the result counts.

adityasharad avatar Jan 29 '20 23:01 adityasharad