concurrent-log-handler
concurrent-log-handler copied to clipboard
Preston-Landers
Time based rotation
As far as I can tell there is no way to configure the rotation by time like TimedRotatingFileHandler, will it be hard to add another handler that will rotate like TimedRotatingFileHandler but with the locking mechanism?
That's not available at the present time, but I don't see why it couldn't be added. I'll take a look when I get a chance. Pull requests are welcome too.
@Preston-Landers Thanks for the reply.
I was thinking of using the built-in TimedRotatingFileHandler handler for the time rotation handling and your ConcurrentRotatingFileHandler for the file locking capabilities.
Do you have any idea how we can do it easily? I'm willing to work on that and provide PR.
@jh11200 this will not use the ConcurrentRotatingFileHandler locking mechanism.
Did you test it with multiple processes writing to the same file?
What is the difference between this and the original time base logger?
@jh11200 我在flask中使用了你的HxTimedRotatingFileHandler 调用为HxTimedRotatingFileHandler (logfile, "midnight", 1, 7) 由于flask在重载代码的机制下为双进程，依然会出现
PermissionError: [WinError 32] 另一个程序正在使用此文件，进程无法访问。: 'D:\\logdev\\normal.log' -> 'D:\\logdev\\normal.log.2019-12-05'
这基本是多个进程占用文件导致文件无法重命名，我不知道你是如何解决的，望告知
Is this going to be added? I was just about to open an issue to ask for a feature request... I need to roll logs daily instead of based on the size. Thanks
Indeed this would be nice to have, did anyone give it a shot? I think starting by extracting most methods (aside _shouldRollover) to a mixin class, and then create the two classes by using multiple inheritance should work (implementing only the specific _shouldRollover methods in both subclasses). But i didn't try yet.
Actually went on and experimented with that, quickly tested with an existing project needing the feature, and it seems to behave as expected, the script that many subprocesses all using the same logger config, and tailing the resulting log file now displays the same result as the console output.
I think also it can be useful having the feature of time rotating for concurrent file handlers. Would be possible to implement it? Thanks!
Hi folks,
Good news for anyone who is interested in this feature. I've written a new implementation of a timed rotating handler that should address the problems I mentioned in PR#48. It's a direct subclass of the base TimedRotatingFileHandler and composes an instance of ConcurrentRotatingFileHandler to reuse locking and other behaviors. It also coordinates the rollover time with other processes. It supports both time-based and size-based rotation (defaults to time-only, every 1 hour, like the base class).
More details can be found in the updated README.md on the branch timed_rotating_v2. You can review PR#55 and leave comments there.
I would be happy if anyone could test this out and provide any feedback. There is already some unit test coverage for the new features but additional testing and validation is always great. If no problems surface within a week or so I may go ahead and publish this as a new version. Please also note that as of this new version we're requiring Python 3.6 or higher.
Use a line in requirements.txt like this to install the branch:
git+https://github.com/Preston-Landers/concurrent-log-handler.git@timed_rotating_v2#egg=concurrent_log_handler
Or you can grab a wheel from here: https://github.com/Preston-Landers/concurrent-log-handler/releases/tag/0.9.23-alpha-1
Note that the original PR for this feature has been rebased to the newest code and is still available on a branch concurrent_timed_rotating_file_handler if you want to test or compare that as well.