Recorder Visits by Device and Browser

[ddfridley]

In order to measure the effect of changes we make to the candidate recording experience, we need to measure how many visits there are to each recorder page, and by what device type (phone or desktop), browser (chrome, safari), and os (Windows, iOS).

This information is available in the logs, but we need a way to process it.

Fortunately, the logs are in Mongo and much of the data is in objects.

app/tools/logwatch.js db $MONGODB_URI will print the last 24 hours of log info to stdout

Here is a small excerpt:

1:25:27 PM node info  [ 'GET',
  '/country:us/state:me/state:u-s-senate-maine/stage:primary/party:democratic-party/2020-07-14',
  'Mozilla/5.0 (Linux; Android 7.0; Alcatel_5044C) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.117 Mobile Safari/537.36',
  { browserConfig:
     '{"os":{"name":"android","version":[7,0],"versionString":"7.0"},"browser":{"name":"chrome","version":[81,0,4044,117],"versionString":"81.0.4044.117"},"type":"phone","model":"","ip":"107.77.165.3"}' } ]
1:25:27 PM node info  [ 'index browser supports ES6' ]
1:25:34 PM node info  [ 'GET',
  '/country:us/state:vt/state-legislative-upper:vermont-state-senate-chittenden-district/stage:primary/party:democratic-party/2020-08-11-recorder-',
  'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36',
  { browserConfig:
     '{"os":{"name":"windows","version":[10,0],"versionString":"10.0"},"browser":{"name":"chrome","version":[83,0,4103,116],"versionString":"83.0.4103.116"},"type":"desktop","model":"","ip":"66.102.8.216"}' } ]
1:25:34 PM node info  [ 'index browser supports ES6' ]
1:25:47 PM node info  [ 'GET',
  '/country:us/state:me/congress:u-s-house-maine-district-2/stage:primary/party:republican-party/2020-07-14',
  'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36',
  { browserConfig:
     '{"os":{"name":"macos","version":[10,12,6],"versionString":"10.12.6"},"browser":{"name":"chrome","version":[84,0,4147,89],"versionString":"84.0.4147.89"},"type":"desktop","model":"","ip":"69.243.54.98"}' } ]
1:25:47 PM node info  [ 'index browser supports ES6' ]
1:25:49 PM browser info  [ '2020-07-27T20:25:47.899Z',
  'client main running on browser',
  { socketId: 'bzAVTVBz-XzLDn5XAAG1', userId: 'anonymous' } ]
1:26:39 PM node info  [ 'GET',
  '/country:us/state:wi/state-legislative-upper:wisconsin-state-senate-district-26/stage:primary/party:democratic-party/2020-08-11',
  'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0',
  { browserConfig:
     '{"os":{"name":"windows","version":[10,0],"versionString":"10.0"},"browser":{"name":"firefox","version":[78,0],"versionString":"78.0"},"type":"desktop","model":"","ip":"50.93.222.117"}' } ]
1:26:39 PM node info  [ 'index browser supports ES6' ]
1:26:41 PM browser info  [ '2020-07-27T20:26:46.903Z',
  'client main running on browser',
  { socketId: '0J2afSbhsmHIrf7-AAG5', userId: 'anonymous' } ]

That file shows how to access that logs data through mongo. The logs use log4js https://www.npmjs.com/package/log4js and a custom file for writing that into Mongo at app/server/util/mongo-logger.js

What we need in the short term is just a report that lists recorder URL, num-visits, [os,browser,type], [os, browser, type], [os, browser, type] where [os,browser,type] is a list showing that info for each visit. The number of visits to a recorder link is 1 to 8 usually. I've never seen as many 16 visits.

In the long term, we can image moving toward an admin console, on a separate server, where people can see lots of information and graphs about undebate data.