Logging 相关

[moooofly]

• [x] State of the event log architecture enhancements
• [x] Logging Best Practices
• [ ] Unified Logging Layer: Turning Data into Action

[moooofly]

State of the event log architecture enhancements

小结：

• 介绍了 syslog 格式扩展，数据结构化的演进；
• 说明了 CEE-enhanced messages and Lumberjack 的起源；
• CEE-enhanced messages 通过定义称为 cookie 的标识符，扩展了日志消息格式，并保证了兼容性；
• Lumberjack 定义了如何格式化上述 cookie 标识符后的 JSON 数据；

---

• event log (syslog) community 的改变 - syslog format extension and structuring syslog data.
• As of today there’s no real standard on how to format and structure data on a syslog message. Every project has its own log message structure and syntax, so we rely on parsers to extract any given data from a log message because the syslog software has no way to do it for us.
• The main idea about structuring syslog messages is to represent them using JSON along with the current free form strings to prevent backward compatibility breakage. To achieve this, we need to normalize and extend this format so that syslog softwares such as rsyslog and syslog-ng can directly understand them. That’s where CEE-enhanced messages and Lumberjack kick in.
• CEE-enhanced messages: The CEE project aims at defining a syntax which extends the current log message format while being compatible with all the currently and widely used log frameworks or the well known glibc’s syslog() call. To achieve this the main idea is to use what is called a cookie before the JSON representation of the data we want to pass to the syslog software.
• The Lumberjack project: So now how do we format the JSON part ? Could we have other types such as booleans and integers directly interpreted by the syslog software ? Well this needs definitions and standardization proposals, that’s what project Lumberjack is for.

[moooofly]

Logging Best Practices

• Local Logging
  • Syslog
  • runit’s svlogd
• Centralized Logging
  • Syslog (Again!)
  • Logstash with Lumberjack
  • Graylog2

• Lumberjack 是一种 log shipper ，log shipper 是 Logstash 中的概念；
• elastic/logstash-forwarder was recently renamed from 'lumberjack' to 'logstash-forwarder' to make its intended use clear. The 'lumberjack' name now remains as the network protocol, and 'logstash-forwarder' is the name of the program. It's still the same lovely log forwarding program you love.
• The lumberjack protocol used by this project exists to provide a network protocol for transmission that is secure, low latency, low resource usage, and reliable.

[moooofly]

Unified Logging Layer: Turning Data into Action

Fluentd 出品的一篇关于如何将 logging 统一化的文章