loguru icon indicating copy to clipboard operation
loguru copied to clipboard

add pls inode check

Open KoBilla opened this issue 5 years ago • 7 comments

https://github.com/Delgan/loguru/blob/93f199042c5612367aab0cbee018961b1748627b/loguru/_file_sink.py#L172

During rotation in multiple process is in all processes decided to rotate. (file.tell()+len(message) > size_limit on same file descriptor although filename was already changed) and processes repeatly rotate a creates new file. Result: (with "rotation": "5 MB" "sink": "app.log") 48K Sep 30 17:49 app.2020-09-30_17-48-39_174339.log 29K Sep 30 17:49 app.2020-09-30_17-48-41_356614.log 50K Sep 30 17:49 app.log

Checking inode (in some interval) is good protection before writing to the wrong file (renamed, deleted)

KoBilla avatar Sep 30 '20 16:09 KoBilla

Hi @KoBilla.

I think multiple processes should not write to the same file, otherwise you will run into problems even with inode checks. It is not possible to build a reliable and robust system without some kind of synchronization between the processes.

If the processes are spawn in Python from the same place using the multiprocessing library, it's possible to ensure that file is properly rotated thanks to the enqueue argument. See this part of the documentation: Compatibility with multiprocessing using enqueue argument.

If the processes are created separately on your system, you need to use a background server in charge of collecting logs and synchronizing file writes. This is possible using a SysLogHandler for example.

Delgan avatar Sep 30 '20 18:09 Delgan

enqueue creates unstable processes management in uwsgi. Unfortunately, for example, restart uwsgi server does not work or restart children when reach max-requests (child freezes at exit - waits for thread)

KoBilla avatar Sep 30 '20 23:09 KoBilla

Coincidentally, the problem with uwsgi has been fixed very recently: #325

Using Loguru v0.5.3, the child processes should no longer hang at exit. If you are still experiencing problems, surely we can investigate it and fix what is wrong.

Delgan avatar Oct 01 '20 08:10 Delgan

Hi. This is next problem. I already use loguru v0.5.3. I'll prepare small example in docker.

KoBilla avatar Oct 01 '20 10:10 KoBilla

Thanks, that will be of great help if I can reproduce the problem!

Delgan avatar Oct 01 '20 10:10 Delgan

loguru.zip All is in README.txt

KoBilla avatar Oct 01 '20 13:10 KoBilla

Great, I can reproduce the problem, thanks!

No kidding, the problem seems to be solved with a single line.

import flask

However, it must be placed in a very specific location. Not at the beginning of the file. Not immediately after the configure(). For this to work, the import must be done after the first logger.info() which is next to the configure().

logger.configure(handlers=[handler1,])
logger.info('start')

import flask

i = 0

I have absolutely no idea why it works this way.

Obviously, this is not a satisfactory solution. No one should have to depend on Flask, let alone where it is imported. This is just a first step to investigate the issue, and hopefully solve it...

Delgan avatar Oct 01 '20 17:10 Delgan