Publish metrics for monitoring

[jasonprado]

Which SIST2 component is your Feature Request related to? Scan/Index

Is your feature request related to a problem? Please describe. If I want to monitor sist2's status in a dashboard tool like Grafana, what is the best way to do that? Right now a tool could examine the .sist2 file and export some metrics from it, but there might be other ways.

What would you like to see happen? The API-serving process could also publish an endpoint /metrics that publishes some basic metrics like number of threads, number of threads in various states, count of items scanned/indexed. This endpoint could be scraped by a tool like Prometheus.

Additional context Prometheus

[simon987]

You could probably use something like Logstash to read from sist2-admin-data/logs/scan-*.log

Or if you are using the executable directly you can use --json-logs to have something more machine-readable