folder2ram icon indicating copy to clipboard operation
folder2ram copied to clipboard

Proxmox VE - Folder2Ram starting too late?

Open mlazzarotto opened this issue 1 year ago • 1 comments

Hi, I've installed Folder2Ram on my Proxmox in an attempt to reduce writes on disk for some folders, including /var/log. My current configuration (excluding commented lines):

tmpfs           /var/log                size=2G
tmpfs           /var/lib/pve-cluster            size=16M
tmpfs           /var/lib/pve-manager            size=1M
tmpfs           /var/lib/rrdcached              size=16M

The problem is that Folder2Ram starts and mounts the folder using tmpfs as expected, but Proxmox keeps writing logs inside /var/folder2ram/var/log/ until the next logrotate rotates the logs. When that happens, the system switches to /var/log.

I think that the issue may be in the order of start of the systemd units. See my systemd-analyze plot picture. systemd_plot

Some info:

  • Proxmox VE 8.1
  • RAM 64GB of which only 41 used (mainly by ARC cache)
  • EXT4 as filesystem on Proxmox disk, ZFS as filesystem for virtual machine disks

Folder2Ram was first configured, then enabled by using -enablesystemd and then I rebooted the system.

mlazzarotto avatar Jun 16 '24 13:06 mlazzarotto

yeah it seems the script is started well after a bunch of things that might be writing logs, probably the first one we can use to order the service is "systemd-journal-flush.service", which is the service that migrates journald's journal to disk after the startup phase where it's in RAM (as there is only RAM before disks are mounted).

This script was developed a few years ago and mostly for OpenMediaVault.

I'll have a look at this when I have some time, probably in a couple weeks. I do have a Proxmox home lab too.

If you want to try changing its startup order on your own, the systemd units starting and stopping folder2ram are

/lib/systemd/system/folder2ram_startup.service
/lib/systemd/system/folder2ram_shutdown.service

You can just edit the files with nano and add Before= or After= or whatever other systemd option to re-order the service you can find, (see the docs https://www.freedesktop.org/software/systemd/man/latest/systemd.service.html ) and then reload the service with systemctl daemon-reload. And then you restart the Proxmox to see if it's working.

this is the current content of the startup, which as you see has a couple After= options to order it after some services (needed for openmediavault)

[Unit]
Description=folder2ram systemd service
After=local-fs.target
After=blk-availability.service
DefaultDependencies=no

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/sbin/folder2ram -mountall
$timeout_line

[Install]
WantedBy=basic.target

bobafetthotmail avatar Jun 16 '24 21:06 bobafetthotmail

after some investigation, the safest measure is to just force a logrotate as part of the folder2ram mount logic, so after it has done the remount of the folders it will call logrotate --force /etc/logrotate.conf to force a rotation, so the overall "solution" is similar to the other bug report about systemd https://github.com/bobafetthotmail/folder2ram/issues/32

I will add this and the other one to the script this or next weekend

bobafetthotmail avatar Aug 01 '24 12:08 bobafetthotmail

I committed the change for this and also for the other bug report. In my testing it seems ok, the logging goes to the tmpfs folder.

bobafetthotmail avatar Aug 04 '24 23:08 bobafetthotmail

similar problem happend with omv-folder2ram on debian 12 :
some services (like apache2) can't start because /var/log is not usable.

olaulau avatar Nov 08 '24 14:11 olaulau