Logger format for relative path to file where log was made and logging package logs to different files.

[prchristie]

I've always found that an extremely useful feature for debugging is having the relative path to the file where the log was made in the log. This allows you to control click the path:line and be immediately brought to the log in vscode (not sure how it works in other ide/editors).

Example of this, we have the structure

setup.py
test_app
|_test_folder
  |_app.py

where app.py has a logger that calls logger.error('Invalid state') on line 50 If the log has the following: [test_app/test_folder/app.py:50] Invalid State and you control click the path you will taken to app.py:50

Using {name} prints the path with . instead of / and I am unsure if there is a way to convert the . to / (which would very adequately resolve the issue) and {file.path} prints an absolute path to the file which also allows for the control click functionality BUT makes the log very long in most cases (think /home/user/workspace/app_root/test_app/test_folder/app.py:50) or even worse if its a package in a virtual environment making the call.

In addition, I think I would have really benefitted from a complete list of formatter options for Loguru that are immediately available and easily findable. Perhaps this was a me issue, but I struggle quite a lot to find https://loguru.readthedocs.io/en/stable/api/logger.html#record which I am still unsure if its complete.

Last thing, I would love to be able to separate my app logs from logs made by packages (like werkzeug completely overrunning my logs) into different files. Given that the point of loguru is to have one formatter, I have a feeling this won't really be possible, but thought I'd ask anyway.

[Delgan]

Hey @prchristie. Sorry for the late answer. I actually like the idea of clickable relative paths, thanks for the suggestion.

Using {name} prints the path with . instead of / and I am unsure if there is a way to convert the . to / (which would very adequately resolve the issue) and {file.path} prints an absolute path to the file which also allows for the control click functionality BUT makes the log very long in most cases (think /home/user/workspace/app_root/test_app/test_folder/app.py:50) or even worse if its a package in a virtual environment making the call.

Do we agree that path should be made relative to the current working directory? Did you have something else in mind?

I also think that path should stay absolute if it's not a subpath of the working directory. This help avoiding technical issues (os.path.relpath() fails for two different Windows drives, symlinks subtleties, etc.). Also, that avoid paths like "../../../../lib/somefile.py" if file is located close to the root folder.

In addition, I think I would have really benefitted from a complete list of formatter options for Loguru that are immediately available and easily findable. Perhaps this was a me issue, but I struggle quite a lot to find https://loguru.readthedocs.io/en/stable/api/logger.html#record which I am still unsure if its complete.

Well, it's near the top of the documentation. You can access it by clicking links in the Readme. I can hardly make it more accessible without burdening the presentation, it seems.

The list you found is complete, though. There are no other attributes for formatting than those mentioned there.

Last thing, I would love to be able to separate my app logs from logs made by packages (like werkzeug completely overrunning my logs) into different files. Given that the point of loguru is to have one formatter, I have a feeling this won't really be possible, but thought I'd ask anyway.

It should be possible using a two sinks with custom filter function. :+1:

[Delgan]

Just sharing a few links for myself:

• __code__. co_filename should always be an absolute path
• How can I include the relative path to a module in a Python logging statement?
• Given two absolute paths, how can I express one of the paths relative to the other?

[prchristie]

Hey, thanks for responding. I agree that it should be relative, as for the most part it is a debugging tool that won't be useful for someone that isn't digging into the code anyway. I haven't considered those absolute path issues you listed. I can imagine a few cases where it would be an issue. I think what you suggest will work well. Alternatively or additionally some solution where we actually replace . with / in the module name then its always relative to the package module layout, but this might have issue with projects that use a src folder as their top level instead of package name.

Well, it's near the top of the documentation. You can access it by clicking links in the Readme. I can hardly make it more accessible without burdening the presentation, it seems.

Ah thats entirely my bad lol.

[wjzhou]

I also think it is very useful if the generated log can be clickable inside vscode

For my use case, I think add an option/(env variable) to output absolute path is sufficient.

When I'm in vscode, (debug mode), I don't really care that the line is too long. (If I can turn on/off this absolute path behavior easily)

[Delgan]

Alternatively or additionally some solution where we actually replace . with / in the module name then its always relative to the package module layout, but this might have issue with projects that use a src folder as their top level instead of package name.

@prchristie Yeah, I would like to avoid special cases where the log messages comes from a library which isn't located in the package itself (might be in src or in the virtual environment).

By the way, until a possible solution is implemented in the next release, you can use patch() as a workaround:

def add_relative_path(record):
    record["extra"]["relative_path"] = record["name"].replace(".", "/") + ".py"

logger.configure(patcher=add_relative_path)

logger.add(sys.stderr, format="{extra[relative_path]}:{line} {message}")

For my use case, I think add an option/(env variable) to output absolute path is sufficient.

@wjzhou Thanks for the feedback. You can configure your handler with "{file.path}"in the format to get the absolute path.

logger.add(sys.stderr, format="{file.path}:{line} {message}")

You can also set the LOGURU_FORMAT environment variable to your preferred format and it will be used by default. :+1:

[wjzhou]

Thank you for the suggestion. works perfectly for my needs.

FYI, for anyone searched this issue. Please use: logger.add(sys.stderr, format="{file.path}:{line} {message}")

And Ctrl-Click on the file will open the line in vscode.