Psi Vesely

Okay, @msheiny fixed this for us on our remote servers, but maybe we should put a note somewhere in the README for the kids who are trying this at home....

@redshiftzero and I discussed this in person for a minute and whether we should increase the `monitored_nonmonitored_ratio` in `fpsd/config.ini`. We decided to leave it for now, but in the future...

[Matthews correlation coefficient](https://en.wikipedia.org/wiki/Matthews_correlation_coefficient) (`sklearn.metrics.matthews_corrcoef`) "is used in machine learning as a measure of the quality of binary (two-class) classifications... generally regarded as a balanced measure which can be used even...

Note if the sorter runs concurrently with the crawler, it will terribly pollute the traces. So either the crawler needs to be stopped, or we need to run the sorter...

I think we should just use another VM for this and a systemd.timer (see https://wiki.archlinux.org/index.php/Systemd/Timers, https://www.freedesktop.org/software/systemd/man/systemd.timer.html, & https://coreos.com/os/docs/latest/scheduling-tasks-with-systemd-timers.html). We could save on VPS money by re-using our database VM as...

@conorsch What do you think of re-using the database server for this purpose? The minimal processing of content means the sorter should be quite safe to run on the database...

Seeing these same errors ``` getFirstPartyURI failed for chrome://browser/content/browser.xul: 0x80070057 [07-18 22:06:29] Torbutton WARN: no SOCKS credentials found for current document. ``` when visiting http://cbw7pgk4jfjl4m6x.onion/, which also stalled out the...

Testing https://github.com/freedomofpress/FingerprintSecureDrop/commit/5802bd36f84b4a51ab08a5ff6becaa25f2726f61 to address this.

Crawls in progress. Will check on them tomorrow morning to see if they failed part-way through or not.

@redshiftzero found that cubie3atuvex2gdw.onion, which redirects to https://another6nnp2ehkn.onion/ (self-signed cert) to have reproduced the error. I'm in the process of refactoring the crawler, but have a couple URLs to add...