RFC: Unified logging library

[nytamin]

About Me

This RFC is opened on behalf of NRK.

Use Case

Background

In the various Sofie apps (Core, Playout Gateway, Mos Gateway, Live Status Gateway, Package Manager etc..) we use slightly different logging methods. Most (all?) are base on the Winston logger, but have different configurations, which results in slight changes in logging between apps and non-type-safe logging methods.

I will be looking into how we can unify logging, and I am looking for input from the Sofie Community.

Initial project plan

(to be revised based on input)

• Make an opinionated, zero-config logging library, based on some other popular logging-library (candidates are Winston, pino, others?).
• The logging library will be exposed either as a part of sofie-core-integration or as it's own npm package.
• The logger should log things in a pre-determined JSON structure. (This is important so that ElasticSearch/Kibana keywords are parsed properly).
• The logger should make logging of Errors easy.
• The logger should provide helper methods for catching application-level events (such as process.on('warning',() ={}) etc).
• The logger should have support for "categories" (ie create a new logger that will prefix logging with catebory breadcrumbs).
• (If needed??) The logger should be able to output logs in non-json (ie human readable) format, when developing.

Community input

I'd like input on things like:

• What is your opinion on havinig a unified logging api/library? Yay or Nay?
• Do you want to be included in discussions regarding this?
• Do you have an preferred in-house way or formatting of logging? (e.g: JSON with certain specified fields?) Do you have any specific requirements from your logging inftrastructure?
• Do you have good experience from other projects to share?
• Any other requirements you'd like to share?
• How is the current logging working for you? Any specific pain points?

---

Depending on the answers, I'll revise the project plan.

Proposal

(No proposal yet, looking for comments)

Process

The Sofie Team will evaluate this RFC and open up a discussion about it, usually within a week.

• [x] RFC created
• [ ] Sofie Team has evaluated the RFC
• [ ] A workshop has been planned
• [ ] RFC has been discussed in a workshop
• [ ] A conclusion has been reached, see comments in thread

[jstarpl]

I would propose we consider LogTape as well as a basis: https://logtape.org/comparison Some of it's advantages that seem nice:

• no dependencies
• small size
• we could use it in libraries as well - "library-first design"

It's not as fully featured as winston, but seems to have the features we use?

[ianshade]

The top qualities for me are:

• Logs easily consumable by the ELK stack
• Error logging more consistent and predictable than it is now

I also think that we would benefit from logs being categorized so that each category can be enabled/disabled independently. Example: Logs of commands sent to devices controlled by the TSR. IMO that should be a category on its own. One might want a production system to log only errors and warnings (definitely not all debug logs, which might be quite detailed and frequent), but also trace crucial communication with external systems. Today this could be implemented on a device basis, but perhaps it could be centralized?

[Julusian]

@jstarpl LogTape looks appealing to me, particularly the library-first aspect.
I think this solves a problem that we encounter; across the various libraries within the mono-repo and the connection libraries, it is hard to do unified logging as we don't want to be importing winston.
As a bonus, LogTape even appears to have a winston adapter, meaning it can with only a couple of lines be piped into winston and pino (which are internally only a couple hundred lines), so this means it doesnt force anyone else to use logtape within their application.
I havent looked over the rest of features, I'm really not sure what we actually use/need from it.

As part of this, we should make sure to use categories (as logtape calls it, child loggers in winston?) everywhere to give the logs more structure. Which as @ianshade asks for, would also allow for configuring different levels based on the source.

The logging library will be exposed either as a part of sofie-core-integration or as it's own npm package. The logger should log things in a pre-determined JSON structure. (This is important so that ElasticSearch/Kibana keywords are parsed properly).

I agree that the setup of the logger should be handled primarily by server-core-integration.
The amount of repeated boilerplate setup code spread across the various gateways makes it very tedious to do anything, so ideally having a function to setup the sink in our standard way (allowing some customisation from gateways, and also for them to simply do it all themselves) would be beneficial