serilog-sinks-file icon indicating copy to clipboard operation
serilog-sinks-file copied to clipboard

Async to remove old log file

Open SpringHgui opened this issue 1 year ago • 6 comments

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. For example, "I'd like to do x but currently I can't because y [...]".

  1. At the beginning of the project, my configuration was like this,72 log files will be retained
 configuration
    .WriteTo.File($"Logs/log-.log",
        rollingInterval: RollingInterval.Day,
        retainedFileCountLimit: 72 ,
        shared: false,
        outputTemplate: temp);
  1. Since the log file is relatively large, compression is used for archiving. my configuration was like this,2 log files will be retained and 70 compressed log files, ArchiveHooks
 configuration
    .WriteTo.File($"Logs/log-.log",
        rollingInterval: RollingInterval.Day,
        retainedFileCountLimit: 2,
        shared: false, hooks: new ArchiveHooks(70),
        outputTemplate: temp);

https://github.com/serilog/serilog-sinks-file/blob/2a61b4bbcb43bf421fc49295462befbb1b6735ec/src/Serilog.Sinks.File/Sinks/File/RollingFileSink.cs#L206

  1. Everything looked fine, but when I upgraded the program to a production environment, the program blocked on startup. Unable to complete startup after several minutes
  2. The first startup after the program upgrade requires compressing 70 files, which takes a lot of time. The program will be blocked until this deleting work is completed

Describe the solution you'd like

Archive files are usually compressed or copied, which takes a lot of time. During this period, the main program should not be blocked from starting. Async to remove old log file supported

https://github.com/serilog/serilog-sinks-file/blob/2a61b4bbcb43bf421fc49295462befbb1b6735ec/test/Serilog.Sinks.File.Tests/Support/ArchiveOldLogsHook.cs#L20

Describe alternatives you've considered A clear and concise description of any workarounds or alternative solutions you've considered.

Additional context Add any other context or screenshots about the feature request here.

SpringHgui avatar Feb 02 '24 01:02 SpringHgui

You mentioned ArchiveHooks - are you using this Serilog plugin? If so, this issue should be opened in that repo, rather than this one.

If I understand correctly, this is a one-time issue the first time you start the app after adding in compression via ArchiveHooks - do you think such a rare event warrants changes? The serilog-sinks-file-archive plugin relies on the (synchronous) OnFileDeleting hook - it has to do the work there (compression), then return control to the main Serilog File sink which will continue to delete the file; I'm unsure how we could prevent blocking.

cocowalla avatar Feb 02 '24 12:02 cocowalla

@cocowalla I looked at the source code, serilog-sinks-file-archive can't be asynchronous, it must be supported by serilog-sinks-file

SpringHgui avatar Feb 02 '24 12:02 SpringHgui

@SpringHgui that's kind of what I was getting at :smile:

But for the issue to be actionable here, might I suggest you couch it in more general terms (rather than related to an issue with one specific plugin), provide a justification, and possibly provide the bones of a solution?

cocowalla avatar Feb 02 '24 15:02 cocowalla

https://github.com/serilog/serilog-sinks-file/blob/2a61b4bbcb43bf421fc49295462befbb1b6735ec/src/Serilog.Sinks.File/Sinks/File/RollingFileSink.cs#L173

change to this?

  Task.Run(() =>
  {
      ApplyRetentionPolicy(path, now);
  });

Increase the speed of program startup and asynchronously archive expiration log file

SpringHgui avatar Feb 03 '24 04:02 SpringHgui