compdemocracy
Clean up server logging
Expected behavior: Logs are useful to look at.
Actual behavior: A simple conversation index or even vote (!) request can produce literally over a thousand lines of logs :scream:
This is because the server:
- logs out intermediate request/response information at several points in the call pathway
- the object (and nested object) key-value pairs are logged out one pair per line, meaning one such object can take hundreds of lines to print out (and aren't even "pretty printed" with indentation that would help parsing nesting structure)
- under any realistic load, this leads to log messages getting interleaved, completely obliterating the intended purpose of being able to sort through kv pairs more easily
- in many place these log messages are preceded by insufficiently descriptive messages like
before,part2, ormike12345
To Reproduce: Fire up the development environment and run a few basic requests.
Screenshots:
Behold the log lines for a single vote (first vote is actually longer)
Additional context: Enough logging solutions (like logentries) support json parsing/processing to make the intended purpose here not only irrelevant, but an actual hindrance (since this feature generally doesn't work if a log message is spread over multiple lines).
Suggested Solution(s): Go through the server code and:
- rewrite the log function that's exploding objects in this fashion
- replace non-descriptive log messages with more useful messages
- (possibly) prune down some of the intermediate logging steps, or set up a verbose logging mode for them
- (possibly) in places where this is used, do some inference to figure out which parameters we actually care about, and either just log those out, or log them out first, prior to the whole dump
- (possibly) do some filtering in the log function to strip out kv pairs we know we don't care about
There are also inconsistencies in the various usage of Winston, console.log, console.err, console.dir and something called yell.
We need to make sure no PIIs are being logged either, including but not limited to user/pass at login time.
Addressed in https://github.com/compdemocracy/polis/pull/1629 and https://github.com/compdemocracy/polis/pull/1638