Is there a way to disable message formatting ?

[gjeusel]

Hi Delgan !

I was wondering if there was a way to disable the message formatting with the args/kwargs ?

from loguru import logger

kwargs = {"a": 1, "b": 2}
logger.info(f"Is this going to be alright ? {kwargs}", additional_infos=kwargs)

Traceback:

Traceback (most recent call last):
  File "/tmp/test.py", line 4, in <module>
    logger.info(f"Is this going to be alright ? {kwargs}", additional_infos=kwargs)
  File "/Users/gjeusel/miniconda3/envs/greenpandas/lib/python3.6/site-packages/loguru/_logger.py", line 1971, in info
    __self._log("INFO", None, False, __self._options, __message, args, kwargs)
  File "/Users/gjeusel/miniconda3/envs/greenpandas/lib/python3.6/site-packages/loguru/_logger.py", line 1948, in _log
    log_record["message"] = message.format(*args, **kwargs)
KeyError: "'a'"

Is it a feature you would be inclined in adding ? Maybe parametrized by an environment variable or via logger.configure ?

Cordially !

[Delgan]

Hi!

Well, you can disable formatting by not providing any argument. If you want some arguments to be added to the extra dict, you can use bind() instead:

logger.bind(additional_infos=kwargs).info(f"Is this going to be alright ? {kwargs}")

Another possiblity is simply to not use f-strings:

logger.info("Is it going to be alright? {additional_infos}", additional_infos=kwargs)

I agree that this may not be the most convenient, since it implies always having to think about that particular case. Especially as auto-formatting is not very useful anymore now that f-strings are widely used.

I'm not a big fan of adding more and more configuration options because it makes the code and the api more complex. But maybe I need to reconsider my position.

[gjeusel]

Thx for your answer ! Indeed, using the bind() to fill up extras might be a solution.

I understand your aversion of adding more configuration. Let's wait and see if this feature is of interest to some more people ?

[Delgan]

I may have some ideas to allow more freedom in configuring the message formatting. I think this feature might be of interest to some people, in your case and others. But I'm not sure of anything, we'll see. I'll keep this issue open in the meantime. :+1:

[kace]

Hi,

I just ran into a similar issue when using an intercept handler to collect logs from pika/pika. The library logs a dictionary at some point, which throws a KeyError exception when I add a keyword argument to the same logging function call.

Your suggestion to use bind() works, but I figured I'd add my use case here anyways.

Thanks for your hard work on Loguru!

[Delgan]

Thanks @kace for the additional use case, it's very helpful to have this kind of example while thinking about a solution.

I plan to propose a way to configure formatting, but I still haven't decided which one. Adding a parameter like logger.configure(autoformat=False) is obviously the easiest, but I'm afraid it opens the door to too many additional parameters. Also, I would like it to be flexible enough to implement customized alternatives like #359. Ideally, one could just patch() the logger using the existing method. But this one is currently called after formatting so it wouldn't work.

Anyway, I am still thinking about this issue. ;)

[soundstripe]

In my case, this happened to me when I combined captureWarnings() with modifying my InterceptHandler to include extra=.

import logging
import loguru
import warnings

logger = loguru.logger

class InterceptHandler(logging.Handler):
    def emit(self, record):
        # Get corresponding Loguru level if it exists
        try:
            level = logger.level(record.levelname).name
        except ValueError:
            level = record.levelno

        # Find caller from where originated the logged message
        frame, depth = logging.currentframe(), 2
        while frame.f_code.co_filename == logging.__file__:
            frame = frame.f_back
            depth += 1

        logger.opt(depth=depth, exception=record.exc_info).log(
            level, record.getMessage(), extra={'more info': 'something else'}  # this line changed from usual to include extra=
        )

logging.captureWarnings(True)
logging.getLogger('py.warnings').handlers = [InterceptHandler()]

if __name__ == '__main__':
    id_ = 23
    warnings.warn(f'problem with something of id: {id_}')

(The above results in an unexpected KeyError because warnings are automatically formatted by Python to include the raw line of code from which they were thrown).

My workaround right now is to use contextualize() instead of adding the extra() manually:

       with logger.contextualize(**{'more info': 'something else'}):  # adds the context data to logged messages here
            logger.opt(depth=depth, exception=record.exc_info).log(
                level, record.getMessage()
            )

[kace]

@soundstripe This is very similar to my use case. You can see my workaround using bind in my comment on another issue. This is especially useful if more info in your code doesn't change between logging calls.

If I remember correctly my original implementation passed the id parameter as a keyword argument to the logging call. I think the documentation mentions that any additional kwargs in this method call are automatically added to the extra's dict.

[mcarans]

Using the standard logger and intercept handler, I get an error with standard_logger.error("hello {fred}"):

Traceback (most recent call last):
  File "/usr/lib/python3.10/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "/usr/lib/python3.10/logging/__init__.py", line 1506, in error
    self._log(ERROR, msg, args, **kwargs)
  File "/usr/lib/python3.10/logging/__init__.py", line 1624, in _log
    self.handle(record)
  File "/usr/lib/python3.10/logging/__init__.py", line 1634, in handle
    self.callHandlers(record)
  File "/usr/lib/python3.10/logging/__init__.py", line 1696, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python3.10/logging/__init__.py", line 968, in handle
    self.emit(record)
  File "<input>", line 20, in emit
  File "/home/mcarans/Programming/VirtualEnvs/hdx-python-utilities-3.10/lib/python3.10/site-packages/loguru/_logger.py", line 2003, in log
    __self._log(level_id, static_level_no, False, __self._options, __message, args, kwargs)
  File "/home/mcarans/Programming/VirtualEnvs/hdx-python-utilities-3.10/lib/python3.10/site-packages/loguru/_logger.py", line 1948, in _log
    colored_message = Colorizer.prepare_message(message, args, kwargs)
  File "/home/mcarans/Programming/VirtualEnvs/hdx-python-utilities-3.10/lib/python3.10/site-packages/loguru/_colorizer.py", line 363, in prepare_message
    tokens = Colorizer._parse_with_formatting(string, args, kwargs)
  File "/home/mcarans/Programming/VirtualEnvs/hdx-python-utilities-3.10/lib/python3.10/site-packages/loguru/_colorizer.py", line 414, in _parse_with_formatting
    obj, _ = formatter.get_field(field_name, args, kwargs)
  File "/usr/lib/python3.10/string.py", line 270, in get_field
    obj = self.get_value(first, args, kwargs)
  File "/usr/lib/python3.10/string.py", line 227, in get_value
    return kwargs[key]
KeyError: 'fred'

[Delgan]

@mcarans Are you able to provide a minimum reproducible example please (eventually in a new ticket)?

I tested it locally and it worked as expected.

[mcarans]

@Delgan It seems to occur when setting record=True in the InterceptHandler (something I put in when I was trying to solve https://github.com/Delgan/loguru/issues/605) ie:

from loguru import logger
import logging

class InterceptHandler(logging.Handler):
    def emit(self, record):
        # Get corresponding Loguru level if it exists
        try:
            level = logger.level(record.levelname).name
        except ValueError:
            level = record.levelno

        # Find caller from where originated the logged message
        frame, depth = logging.currentframe(), 2
        while frame.f_code.co_filename == logging.__file__:
            frame = frame.f_back
            depth += 1

        logger.opt(
            colors=True,
            record=True,
            depth=depth,
            exception=record.exc_info,
        ).log(level, record.getMessage())


logging.basicConfig(handlers=[InterceptHandler()], level=logging.NOTSET)

standard_logger = logging.getLogger("test")
standard_logger.error("hello {fred}")

[Delgan]

@mcarans When record=True Loguru expects the logged message to contain the "{record}" formatting field, e.g:

logger.opt(record=True).info("Logging some message from thread {record[thread].id}")

To do so, Loguru will call message.format(...). In your case, the message is "hello {fred}" but there is no fred keyword argument to be fetched to format the message, hence the error.

For this reason I would advice against using record=True when you don't control the formatted message.

[fernhtls]

Hi @Delgan , I've just started using loguru with FastAPI and it's very good, but I bumped into the same issue above with our InterceptHandler, and in some cases we have dictionaries being pushed to the log message, and I could not bypass the message.format(...), using any configuration above. Any plans to include some bypass config? Thanks for the nice work with loguru!

[fmmoret]

+1

[mastern2k3]

I want to stress the problem with this automatic format behavior. Say we want to log, for whatever reason, data input by users.

There would be no way for me to do this without risking a crash when any string containing curly brackets is passed.

[Delgan]

@mastern2k3 I'm not sure to understand.

logger.info("User data: {}", input())

That would work fine and wouldn't not crash regardless of user data containing curly brackets or not.

[mastern2k3]

@Delgan you are right, but, This means I'm prevented from using f-string formatting as the standard throughout my system, and need to keep asking myself whether a certain variable might contain curly braces or not, and use one or the other depending on which case.

[Delgan]

@mastern2k3 You can use f-strings as long as you do not call the logging method with formatting arguments:

logger.info(f"User data: {user_data}")                      # Good.
logger.info(f"User data: {user_data}", data=user_data)      # Bad.
logger.bind(data=user_data).info(f"User data: {user_data})  # Good.

I got your point, though. I plan to remove automatic formatting in favor of f-strings.

[mastern2k3]

@Delgan Thanks! I didn't notice this is triggered only when there's extra data, that helps a bit.

We often use named parameters after the message (as an f-string) to add attributes to the extras so it ends up in our structured logs, this is what gets me.

[v1ack]

@Delgan hello, I faced the same issue. In my case I want to do my own format function of json serialisation to elasticsearch json format

[v1ack]

No, not the same issue) I found a solution for my case by using message.record in sink function

[mcarans]

@Delgan Is removing automatic formatting in favor of f-strings still planned?

[Delgan]

@mcarans Yeah, this is something I would like to do, but I don't have an ETA or technical solution for the transition yet.