Examples of remapping fields?

[tejasmanohar]

If one wanted to apply custom transformations like remapping a field message -> msg, how would they go about doing that? This is the simplest and most common type of transformation so I think an example in the README would be great.

[rsyring]

I was just wondering the same thing. I want the asctime logging field to show up under a timestamp key in the json.

[fabito]

Need the same for: levelname -> severity Looking at the unit tests, seems like the way to go is using a custom JsonFormatter

def testJsonCustomLogicAddsField(self):
    class CustomJsonFormatter(jsonlogger.JsonFormatter):

        def process_log_record(self, log_record):
            log_record["custom"] = "value"
            # Old Style "super" since Python 2.6's logging.Formatter is old
            # style
            return jsonlogger.JsonFormatter.process_log_record(self, log_record)

    self.logHandler.setFormatter(CustomJsonFormatter())
    self.logger.info("message")
    logJson = json.loads(self.buffer.getvalue())
    self.assertEqual(logJson.get("custom"), "value")

Maybe something like this ?

class CustomJsonFormatter(jsonlogger.JsonFormatter):

    def process_log_record(self, log_record):
        log_record["severity"] = log_record.pop('levelname', None)
        return jsonlogger.JsonFormatter.process_log_record(self, log_record)

[schlitzered]

i created this pull request that deals with this problem.

it allows to to replace field names by passing a dict to rename_fields to the JsonFormatter class like this

fmt = jsonlogger.JsonFormatter(rename_fields={'message': '@message'})

[evandam]

Hi folks,

I'm trying to use rename_fields like this but I can't seem to get it to work as expected?

I want to rename things like name and levelname similar to @fabito

jsonlogger.JsonFormatter(rename_fields={"levelname": "severity", "name": "logger"})

I'm subclassing the formatter anyway and was able to add the following:

def add_fields(self, log_record, record, message_dict):
    super().add_fields(log_record, record, message_dict)
    if self.rename_fields:
        for field in record.__dict__:
            if field in self.rename_fields:
                log_record[self.rename_fields[field]] = getattr(record, field)
                log_record.pop(field)

[wojciechwroblewski]

There are few gotchas about the rename part. The case @evandam described will work fine, if format string contains relation to renamed fields, e.g.:

jsonlogger.JsonFormatter("%(levelname)s %(name)s", rename_fields={"levelname": "severity", "name": "logger"})

produces following output:

{"severity": "ERROR", "logger": "root"}

The use of name in format string may be misleading in this context, as we expect the key to be called logger after rename.

I recently run into similar issue trying to rename exc_info field like so:

jsonlogger.JsonFormatter("%(exc_info)s", rename_fields={"exc_info": "exception"})

This results in traceback being logged under exc_info key:

{"exception": ["<class 'Exception'>", "", "File "/.../main.py", line 14, in \n raise Exception"], "exc_info": "Traceback (most recent call last):\n File "/.../main.py", line 14, in \n raise Exception\nException"}

In this case, "hacking" format string does not work as expected, as we get exc_info logged anyway aside exception. Also its formatting differs - it's a list instead of string.

code snippet:

import logging
from pythonjsonlogger import jsonlogger

logger = logging.getLogger()

logHandler = logging.StreamHandler()
formatter = jsonlogger.JsonFormatter("%(exc_info)s", rename_fields={"exc_info": "exception"})
logHandler.setFormatter(formatter)
logger.addHandler(logHandler)

try:
    raise Exception
except Exception:
    logger.exception("")

[timurstrekalov]

For posterity, an excerpt of how to configure field remapping in JSON that works:

  "formatters": {
    "json": {
      "()": "pythonjsonlogger.jsonlogger.JsonFormatter",
      "format": "%(asctime)s %(levelname)s %(name)s %(message)s",
      "rename_fields": {
        "asctime": "timestamp",
        "levelname": "level",
        "name": "logger",
        "message": "msg"
      }
    }
  }

[raymondberg]

For future hopeful config users. logging.config.fromConfig() cannot build a formatter with renamed fields. The config file only supports a few very standard positional args including format.

https://github.com/python/cpython/blob/da397194832c4b8db8446af42919d8ad47b3cb4a/Lib/logging/config.py#L119

    for form in flist:
        sectname = "formatter_%s" % form
        fs = cp.get(sectname, "format", raw=True, fallback=None)
        dfs = cp.get(sectname, "datefmt", raw=True, fallback=None)
        stl = cp.get(sectname, "style", raw=True, fallback='%')
        c = logging.Formatter
        class_name = cp[sectname].get("class")
        if class_name:
            c = _resolve(class_name)
        f = c(fs, dfs, stl)

[Saphyel]

What's the new way to rename_fields from a config file then?