Use different log levels for different loggers

[Snawe]

Hi!

I wanted to try out loguru instead of the standard logging. However, I have one issue where I kinda find no solution to it.

My main issue is that I have multiple loggers. I use one for my application and two more for libraries (like logging.getLogger("uvicorn.access")). My main application logs on INFO, while for the libraries I use WARNING.

So for example currently I have

• My application logging to console with INFO and to file (application.logs) with INFO
• the uvicorn logs are with WARNING to console and INFO to uvicorn.logs
• Both have different formatter.

I tried with loguru something like

logger.add(sys.stderr, level=logging.INFO)
logger.add("logs/application.log", level=logging.INFO)

for _log in ["uvicorn.access", "uvicorn", "uvicorn.error", "fastapi"]:
    # Found here on some issue I guess.
    _logger = logging.getLogger(_log)
    _logger.handlers = [InterceptHandler(_log)]
    logger.add(f"logs/{_log}.log", level=logging.INFO)
    logger.add(sys.stderr, level=logging.WARNING)

but thats obviously not possible.

So basically I want to have different log levels on different loggers. I tried the filter but couldn't make it work. I either get no logs or all logs in all files. In addition I would also like to be able to:

• Put each application/library to it's own log file
• Have different extra/format for each application/library

Is this in general possible with loguru?

[Delgan]

I think this is possible although this may require some tweakering of your configuration.

For example, I see that you calling logger.add(sys.stderr) multiple times which is likely a mistake, as it would cause duplicated logs.

Which kind of filter did you try?

For example, if you want to log to the console with INFO by default, but with WARNING for Uvicorn, you could use something like that:

filtering_dict = {
    "": "INFO",
    "uvicorn": "WARNING",
    "fastapi": "WARNING",
}

logger.add(sys.stderr, filter=filtering_dict)

[Snawe]

Hey!

Thanks for your reply! Sorry, I am a bit busy atm. I will try out your posted filter and provide more information once I visit that topic again. Hopefully within the next few weeks.

[qiuxiaomu]

Hey Delgan, thanks for such a great logging facility; I happen to have stumbled on the issue, i.e. having a log-to-file handler and stderr handler wanting to log different level of severity. the stderr is mostly informative so INFO and above can do but file handler is meant for analysis so much more is needed.

on a side note, I wasn't able to use {extra[name]} in file name, whereas in formatter worked fine.

I've tried, so far, these two options:

  logger.add(
        sink=sys.stderr,
        format=log_format,
        level="INFO",
        backtrace=True,
        diagnose=True,
        filter={"": "INFO"}
    )

  logger.add(
        sink=sys.stderr,
        format=log_format,
        level="INFO",
        backtrace=True,
        diagnose=True,
        filter=lambda record: record['level'].name in ["INFO", "WARN", "SUCCESS", "ERROR", "CRITICAL"]
        #  filter=lambda record: record['level'].name == "INFO"
    )

Neither worked for me.

Here's my code for the context:

def get_logger(**kwargs):

    logging_root = kwargs.get('root_dir', None)
    log_level = kwargs.get('log_level', 'DEBUG')
    logger_name = kwargs.get('name', 'temp')

    if logging_root is None:
        warnings.warn("no logging file destination provided, logging default to screener log.")
        logging_root = log_settings()['screen_log_root']

    logger.configure(extra={"name": logger_name})
    log_format = "{time} | {level} | {extra[name]} | {message}"
    log_file_name = "data_{time}.log"
    # Cause KeyError: extra: log_file_name = "{extra[name]}_{time}.log"
    log_file_path = f"{logging_root}{log_file_name}"

    # log to file for later analysis
    logger.add(
        sink=log_file_path,
        format=log_format,
        level=log_level,
        colorize=True,
        rotation="10 MB",
        retention="2 months",
        backtrace=True,
        diagnose=True
    )

    # log to stderr
    logger.add(
        sink=sys.stderr,
        format=log_format,
        level="INFO",
        backtrace=True,
        diagnose=True,
        filter={"": "INFO"}
    )

return logger

Thanks a lot for the help!

[Delgan]

Sorry for the late answer @stucash.

Your code is working fine for me.

logger = get_logger(root_dir="logs/", log_level="DEBUG", name="bench")

logger.info("This is an info message")
logger.debug("This is a debug message")
logger.error("This is an error message")

This causes the following lines to be printed on the stderr:

2024-10-05T23:01:35.085778+0200 | INFO | bench | This is an info message
2024-10-05T23:01:35.086033+0200 | ERROR | bench | This is an error message

And the following lines to be written in the file:

2024-10-05T23:01:35.085778+0200 | INFO | bench | This is an info message
2024-10-05T23:01:35.085948+0200 | DEBUG | bench | This is a debug message
2024-10-05T23:01:35.086033+0200 | ERROR | bench | This is an error message

Note, however, that you must call get_logger() only once in your program. If get_logger() is called at multiple places, then the global logger will be re-configured multiple times, which could lead to unexpected behavior.