pgcli icon indicating copy to clipboard operation
pgcli copied to clipboard

Implement daily log rotation with improved log management

Open DiegoDAF opened this issue 4 weeks ago • 2 comments

Summary

This PR implements automatic daily log rotation and improves the default log location for pgcli.

Changes

1. Daily Log Rotation

  • Implements automatic log rotation at midnight using TimedRotatingFileHandler
  • Keeps 30 days of log history automatically
  • Log files are named with the format: pgcli.YYYY-MM-DD.log
  • Old logs are automatically cleaned up after 30 days

2. Improved Default Log Location

  • New default location: /var/log/pgcli/pgcli.log (system-wide logging)
  • Automatic fallback to ~/.config/pgcli/log if user lacks system permissions
  • Creates directories automatically if they don't exist

3. Updated Documentation

  • Updated pgclirc configuration file with new log behavior
  • Added Internal section to changelog.rst

Technical Details

Before:

  • Single log file that grew indefinitely
  • Default location: ~/.config/pgcli/log
  • Manual cleanup required

After:

  • Automatic daily rotation at midnight
  • 30-day retention policy
  • Better default location with fallback
  • Log format: pgcli.YYYY-MM-DD.log

Files Modified

  • pgcli/main.py: Added logging.handlers import and updated initialize_logging() function
  • pgcli/pgclirc: Updated log_file documentation
  • changelog.rst: Added entry for log rotation feature

Testing

Tested manually:

  • Log rotation works correctly at midnight
  • Automatic directory creation works
  • Fallback to user directory works when lacking system permissions
  • Old logs are properly cleaned up after 30 days
  • Log file naming follows the new format

Backward Compatibility

  • Fully backward compatible
  • Existing log files are not affected
  • Users with custom log_file settings in config will continue to work as before
  • Only affects default behavior when no custom log location is specified

Benefits

  1. Better disk management: Automatic cleanup prevents disk space issues
  2. Easier troubleshooting: Date-based log files make it easier to find relevant logs
  3. System-friendly: Uses standard /var/log location when possible
  4. No breaking changes: Respects user configuration and provides sensible fallback

Made with ❤️ and 🤖 Claude Code

DiegoDAF avatar Dec 04 '25 18:12 DiegoDAF

Thanks, no worries, I'm not expecting all the ppl agree with my ideas. I'll explain myself a little bit more. The PRs are my gratitude to the project. I found pgcli to be an incredible tool for my daily work, automations, etc.


Now going to the code:

Personally, I don't like logs in the user home folders, growing forever, in an unmanageable giant file. And ofc I hate the log files in my home backups. So/var/log is the standard place for apps (in Linux, right, I have not tested in Windows or Mac). Based on pgAdmin, for example, it saves the logs in the user's home folder with rotation, ofc, but not with a limit.

my pgdamin home fodler right now: image

Ofc, I was thinking of my objectives when I made this proposal, and now, under review is different.

  1. The main issue with the location is the permission, as you said, and we don't want to add an extra problem at the deployment time. Its logic has the log in the app folder by default. Could we add a log_folder parameter as an option?
  2. The rotation is to keep it in small files; daily is the usual, not a real deal.
  3. The hardcoded 30 days is arbitrary; really, it must be removed, and the user should administer the log files. /var/log is pre-configured by the logrotate system on Linux.

Please tell me what you think, I can change fix the code.

DiegoDAF avatar Dec 05 '25 19:12 DiegoDAF

@DiegoDAF Well, I was happy to see a new feature in pgcli. :) But @dbaty has a good point. It's the code we have to maintain. And we already find it very difficult to juggle contributing to pgcli in our "free time" (yeah right... free time? who has that?) with our day jobs and family responsibilities.

Still, I think that supplying a custom logging configuration is a reasonable ask. But I would do it in a declarative way, instead of the imperative way - less code to maintain.

  1. User would configure a location of logging_config in pgclirc.
  2. If that file is configured and present, we would initialize the logger from dict stored in that file.
  3. Otherwise, proceed with our regular initialize_logging.

WDYT?

P.S. I agree with @dbaty on /var/log/pgcli.

j-bennet avatar Dec 10 '25 01:12 j-bennet

Closing this PR in favor of #1547 which includes the complete implementation with proper test coverage and documentation.

DiegoDAF avatar Dec 16 '25 18:12 DiegoDAF