fides icon indicating copy to clipboard operation
fides copied to clipboard

Add table and models to support system scan history

Open allisonking opened this issue 2 years ago • 1 comments

Is your feature request related to a specific problem?

We would like to persist a user's system scan history.

Describe the solution you'd like

We should have a table with each time a user initiates a system scan. We'll also need the associated models.

Describe alternatives you've considered, if any

A description of any alternative solutions or features you've considered.

Additional context

https://github.com/ethyca/fidesctl-plus/issues/188

allisonking avatar Oct 11 '22 21:10 allisonking

For now I think a single table, plus_system_scans, would suffice. It should support at least the following columns:

  • id - psql default primary key, identifies unique scans
  • created_at - datetime when the scan was initiated
  • updated_at - datetime when the record was last modified
  • status - string representing one of a few possible states of the scan (scanning, classifying, writing results (?), error, complete)
  • error - nullable string, contains an error that prevented the scan from completing successfully, when applicable
    • Sensitive info or system details must be pruned before writing to this column
  • results - JSON blob containing the list of System resources discovered by the scan (with sorted keys, if possible, to support future diff-ing)

Optionally we may find it beneficial to include the following additional columns, although there is no immediate use case:

  • system_count - int representing the number of systems discovered
  • classified - boolean representing whether the classifier was used as part of the scan
    • We should think about a better name for this column, as we don't want to imply that any scan record contains sensitive information
  • stats - JSON blob containing arbitrary statistics that may be fun to display in the UI
    • "most used data_category" (or data_categories, if we want to display a list in descending order)
    • "most ingressed system"
    • "most egressed system"
    • ?

PSalant726 avatar Oct 11 '22 22:10 PSalant726

Completed in #1554.

PSalant726 avatar Oct 27 '22 17:10 PSalant726