Interop with Rich, and capturing exceptions

[autoferrit]

I have my logging setup to use rich's nice error handling with loguru. Though i have a few minor issues. When I am using a try/except block I can easily enough import the console I setup in my logging.py to spit out a nicely formatted error message. But when another exception is raised somewhere in the code, it is not using rich's output.

Does anyone know if there is a better way to set this up?

here is my logging.py

# Futures
from __future__ import annotations

# Standard Library
import logging

from pprint import pformat

# Third Party
import loguru

from loguru import logger as loguru_logger
from loguru._defaults import LOGURU_FORMAT  # noqa
from rich.console import Console
from rich.logging import RichHandler


BO_FORMAT = (
    "\n<blue>{name}</blue>:<cyan>{function}</cyan>:<yellow>{line}</yellow>\n"
    "<level>{message}</level>"
)


class InterceptHandler(logging.Handler):
    """Logging intercept handler."""

    def emit(self, record: logging.LogRecord) -> None:  # pragma: no cover
        """Log the message."""
        try:
            level: int | str = logger.level(record.levelname).name
        except ValueError:
            level = record.levelno

        frame, depth = logging.currentframe(), 2
        while frame.f_code.co_filename == logging.__file__:
            frame = frame.f_back  # type: ignore
            depth += 1

        log = logger.bind(request_id="app")
        log.opt(depth=depth, exception=record.exc_info).log(level, record.getMessage())


def _format_record(record: dict[str, dict[str, str]]) -> str:
    """
    Set custom format for loguru loggers.

    Uses pformat for log any data like request/response body during debug. Works with
    logging if loguru handler it.

    """
    format_string = BO_FORMAT

    if record["extra"].get("payload") is not None:
        record["extra"]["payload"] = pformat(
            record["extra"]["payload"], indent=4, compact=False
        )
        format_string += "\n<level>{extra[payload]}</level>"

    format_string += "{exception}\n"
    return format_string


console = Console(
    force_terminal=True,
    force_interactive=True,
    # width=120,
    emoji=True,
    markup=True,
    color_system="truecolor",
    stderr=False,
)


def _get_logger() -> loguru.Logger:
    """Set up the logger."""
    # set format
    loguru_logger.configure(
        handlers=[
            # {"sink": sys.stdout, "level": logging.DEBUG, "format": _format_record}
            {
                "sink": RichHandler(
                    markup=True,
                    rich_tracebacks=True,
                    tracebacks_show_locals=True,
                    console=console,
                ),
                "format": _format_record,
            }
        ]
    )

    # works with uvicorn>=0.11.6
    loggers = (
        logging.getLogger(name)
        for name in logging.root.manager.loggerDict  # type: ignore # noqa
        if name.startswith("uvicorn.")
    )
    for uvicorn_logger in loggers:
        uvicorn_logger.handlers = []

    logging.getLogger("uvicorn").handlers = [InterceptHandler()]

    return loguru_logger


logger = _get_logger()

Is there anything I could be doing differently so that when any exception comes up, rich can catch it and work it's magic on the output for exceptions?

[astromancer]

I used the following:

from loguru import logger

# Pretty stack traces
try:
    import rich

    def catch(func):
        def runner(*args, **kws):
            try:
                return func(*args, **kws)
            except Exception as err:
                logger.error('{} in main:\n{}', type(err).__name__, format_traceback())
                raise

        return runner

    def format_traceback():
        console = rich.get_console()
        with console.capture() as capture:
            console.print_exception(show_locals=True)
        return capture.get()

except ImportError:
    catch = logger.catch(reraise=True)

Then use like:

catch(my_flaky_function)(*args, **kws)

# or as a decorator
@catch 
def flaky():
    1 / random.choice([0, 1, 2])

[ryaminal]

@Delgan, i'm thinking of picking this up and working on it, as it seems nice(and then i wouldn't "have" to use another dependency).

Curious what the thoughts are on this. I imagine this could be done as an "extra" e.g. pip install loguru[rich] perhaps? and if that extra is there then enable rich output?

And if an extra is desired, will loguru always use the RichFormatter/RichHandlerSink or does this need to be enabled with a call or option or something? I would certainly prefer a

try:
  import rich
  setup_rich_formatter()
except ImportError:
  pass

sort of thing but i also don't want to impose rich logging on everyone just because rich is "importable".

[Delgan]

Hi.

Sorry @autoferrit and @astromancer, I didn't investigate this issue earlier.

Using the rich formatter for exception logged by loguru should be done by following this recipe from the documentation. It simply involves using a custom format function like so:

import io
import sys
from loguru import logger
from rich.console import Console
from rich.traceback import Traceback

def rich_formatter(record):
    format_ = "<blue>{name}</blue>:<cyan>{function}</cyan>:<yellow>{line}</yellow> <level>{message}</level>\n"
    if record["exception"] is not None:
        output = io.StringIO()
        console = Console(file=output, force_terminal=True)
        traceback = Traceback.from_exception(*record["exception"])
        console.print(traceback)
        record["extra"]["rich_exception"] = output.getvalue()
        format_ += "{extra[rich_exception]}"
    return format_

logger.remove()
logger.add(sys.stderr, format=rich_formatter)

@logger.catch
def divide(a, b):
    a / b

divide(1, 0)

Output:

__main__:<module>:25 An error has been caught in function '<module>', process 'MainProcess' (30883), thread 'MainThread' (140601503160128):
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/delgan/programming/loguru/loguru/_logger.py:1281 in catch_wrapper                          │
│                                                                                                  │
│   1278 │   │   │   │   │                                                                         │
│   1279 │   │   │   │   │   def catch_wrapper(*args, **kwargs):                                   │
│   1280 │   │   │   │   │   │   with catcher:                                                     │
│ ❱ 1281 │   │   │   │   │   │   │   return function(*args, **kwargs)                              │
│   1282 │   │   │   │   │   │   return default                                                    │
│   1283 │   │   │   │                                                                             │
│   1284 │   │   │   │   functools.update_wrapper(catch_wrapper, function)                         │
│                                                                                                  │
│ /home/delgan/programming/loguru/d.py:23 in divide                                                │
│                                                                                                  │
│   20                                                                                             │
│   21 @logger.catch                                                                               │
│   22 def divide(a, b):                                                                           │
│ ❱ 23 │   a / b                                                                                   │
│   24                                                                                             │
│   25 divide(1, 0)                                                                                │
│   26                                                                                             │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ZeroDivisionError: division by zero

Thanks also @ryaminal for your proposal. I don't want to add a dependency to rich, not even an optional one. However, if you create a separate package such as loguru-rich for example, I would mention it in the documentation.

[ryaminal]

@Delgan, can you help me understand why a dependency(optional or not) on rich is not desired?

the huge draw to loguru, for me, is that i can just do from loguru import logger as log and not have to think about anything else. rich is a fantastic tool for helping with this and seems like it would be a better choice than just plain colorama.

i understand i can do the import above and still have great output but rich is great for lots of reasons and exists in most projects in data science and other realms already.

[Delgan]

@ryaminal Adding dependencies to a library must be carefully considered for several reasons.

First, it increases the risk of conflict between packages and add complexity in the installation process. Requiring rich not only brings a new library, but also all its dependencies, recursively. This can have undesirable effects for the end user, when two incompatible versions of the same library are required.

A logging library, like Loguru, is a fairly generic tool. It is suitable for virtually any project. Therefore, it is crucial to limit its scope and keep its footprint small. It is undesirable for a project to incur indirect dependencies on rich solely for logging purposes. This not only burdens the installation process, but also introduces additional constraints, increases the package's size, and may lead to compatibility issues – all aspects that contradict the aim for a lightweight library.

Actually, I already had requests to reduce dependencies in the past: https://github.com/Delgan/loguru/issues/95

Apart from that, it also adds maintenance overhead. By minimizing dependencies, the maintenance burden on the Loguru project is reduced. Fewer dependencies mean fewer potential issues with compatibility, updates, and conflicts with other libraries.

Technically, depending on rich wouldn't be enough. We'd also have to extend Loguru's API and add an option such as use_rich_exception_formatting to logger.add(). This is because a user may have installed rich independently, and perhaps doesn't want it to interfere with Loguru's exception formatting.

While Rich is undoubtedly a powerful tool and is widely used, I think Loguru should remain independent of it.

Loguru is designed as a minimalist yet highly flexible library. This is achieved by allowing many aspects to be configured, without embedding more code than necessary. I'm thinking about simplifying the configuration of formatting exception, though. However, if you want to be able to import and use Loguru with Rich without any configuration whatsoever, I'd recommend implementing it in a new library.

[ryaminal]

@Delgan thanks for taking the time to answer that so thoroughly. perhaps add this to a wiki or something as i'm sure i won't be the last moocher to ask about rich. :)

thanks for making an awesome library. i usually hesitate to use anything but print in small python utilities/scripts because i always forget the incantation to get logging going.

[Delgan]

Thanks @ryaminal, glad you like Loguru, I created it for the same reasons you raise. ;)