operating-system icon indicating copy to clipboard operation
operating-system copied to clipboard

How can i increase my swap file size?

Open basdejong95 opened this issue 4 years ago • 59 comments

HassOS release with the issue:

HassOS 4.15

Supervisor logs:

Journal logs:

Kernel logs:

Description of problem:

Hi everybody,

I am using home assistant on a RPI 4 with 1GB of ram. My swap file is only 230mb and fills up pretty fast, in the last week or so i had home assistant crashing a lot and i noticed my swap file is almost pretty full. While reading about this there are pretty clear instructions on the internet on how to increase the swap file on a standard RPI, but i cannot find it anywhere for HASSOS. Does anybody know a command or a setting i can change to increase this? I have a 128GB SD card which is A2 class so i dont mind creating a swap file of 1-2GB.

Thank you!

basdejong95 avatar Nov 07 '20 14:11 basdejong95

Currently Home Assistant OS does not really support that use case. This is also taking a strain on your SD card as swap usually generates quite a bit of read and writes. Since we already have users which have problems with SD cards wearing out, in a lot of cases using swap on SD card does not make sense.

By default HAOS configures a part of the RAM as compressed swap storage (using zram). This helps quite a bit for boards with low memory.

That said, when you access HAOS directly (via SSH port 22222), you can create a swapfile and enable it as swap, but the swap will not be automatically re-enabled after a reboot:

# fallocate -l 4G /mnt/data/test.swap
# mkswap /mnt/data/test.swap
# chmod 0600 /mnt/data/test.swap
# swapon /mnt/data/test.swap

agners avatar Nov 09 '20 11:11 agners

hi, my hassos lives in a rpi3b+ with a 120Go ssd connected via usb i moved the data partition on the ssd my memory is at 80% and swap at 100% i don't have so much devices and entities (357) (discovered HA 2 weeks ago) using rfxtrx, z-wave, zigbee top says that homeassistant is consuming 31%, mysqld 16%, systemd-journald 5%, node-red 5% and supervisor 2.5% i succesfully added a 4G swap file as describe above, i'll see if it prevent my instance to freeze is there any way to make it permanent ? i mean active on reboot i saw that there is nothing on /etc/fstab i know nothing about buildroot thanks

bachy avatar Nov 11 '20 10:11 bachy

Do yourself a favor and switch over to an RPi4 with more than 1GB of RAM. You will only waste time with you trials on 1GB. The difference that more RAM makes is overwhelming.

Jpsy avatar Nov 11 '20 12:11 Jpsy

Yes i know, unfortunatly my z-stick gen5 is not compatible with rpi4 and i'm lazzy to rebuild all my zwave mesh (qubinos wire pilote on the wall behind heaters)

bachy avatar Nov 11 '20 12:11 bachy

Do yourself a favor and switch over to an RPi4 with more than 1GB of RAM.

Better yet, grab an ODROID N2 or N2+

frenck avatar Nov 11 '20 13:11 frenck

Do yourself a favor and switch over to an RPi4 with more than 1GB of RAM. You will only waste time with you trials on 1GB. The difference that more RAM makes is overwhelming.

Home assistant says on the getting started website that 1GB ram should work... And it would work if my swap file could be bigger. However there is no way to permanently change this, only the commands from @agners are a manual solution for now?

basdejong95 avatar Nov 12 '20 14:11 basdejong95

image

this is my swap file within 1 day of rebooting

basdejong95 avatar Nov 12 '20 14:11 basdejong95

@agners that standard solution doesn't seem to work... see the error which suggests that BusyBox and not the full version of swapon is what HassOS seems to be using. Error you get is invalid argument for swapon. FYI, swapon -a ran fine but did not use the newly created swap file either. With my SSD plugged into the RPI3B+, it'd be nice to be able to control the default swap behavior in HassOS. image

JZ-SmartThings avatar Nov 14 '20 23:11 JZ-SmartThings

for me image

QbaF avatar Dec 04 '20 16:12 QbaF

Currently Home Assistant OS does not really support that use case. This is also taking a strain on your SD card as swap usually generates quite a bit of read and writes. Since we already have users which have problems with SD cards wearing out, in a lot of cases using swap on SD card does not make sense.

By default HAOS configures a part of the RAM as compressed swap storage (using zram). This helps quite a bit for boards with low memory.

That said, when you access HAOS directly (via SSH port 22222), you can create a swapfile and enable it as swap, but the swap will not be automatically re-enabled after a reboot:

# fallocate -l 4G /mnt/data/test.swap
# mkswap /mnt/data/test.swap
# chmod 0600 /mnt/data/test.swap
# swapon /mnt/data/test.swap

This works like a treat! On my ssd that should be perfect to prevent filling up the ram completely.

Any idea on how to make this persistent?

capstan1 avatar Dec 08 '20 19:12 capstan1

I do understand what @agners say and I do agree that if you use an SD card increasing the swap is not great but if you use an SSD then shouldn't be a problem.

Since few months ago my pi3b+ is getting very slow and HA ends up not responding. Restart fixes it and then after 1 week or so again the same thing. I’ve added few checks and realized that it may be related with memory… once swap hits close to 100% and RAM 90% this happens. I do not have that many addons and it was working ok before… there are few discussion on potential memory leaks but anyway it is not the purpose of this topic.

I’m running an SSD with my pi so raising the swap size could reduce or even solve some of these issues. The default setting is just 25% of the real device memory (in this case 256Mb) and I think that it is probably having in SDcards in mind (and not SSD)

I’ve managed to change it using the commands below and so far so good… the problem is that this is not persistent (and after restart goes back to original swap size). As far as I understood I would need to change the /usr/libexec/hassos-zram or /usr/lib/systemd/system/zram-swap.service but the filesystem is read-only. Is there any other way to do this?

swapoff /dev/zram0
zramctl --reset /dev/zram0
zramctl --find --size 1024M
mkswap /dev/zram0
swapon /dev/zram0

dpjrodrigues avatar Dec 09 '20 09:12 dpjrodrigues

Just a quick update to mention that increasing the swap size is working well so far. Now the RAM is stable 65% - 70% and the swap is between 30% - 35% so not that much from the forced 25% (the problem is that this configuration is lost when restarted)

dpjrodrigues avatar Dec 11 '20 10:12 dpjrodrigues

I got the exact same problem. It would be great to edit somenthing on hassio config file prior to boot to set up as SSD.

marcoevcom avatar Dec 31 '20 04:12 marcoevcom

I found a way to make this "persistant" using the SSH & Web Terminal Addon. When disabling protection mode for the addon you can simply place the swape file inside the Core folders (for example "/config" oder "/config/www") and issue the swapon command as initial command of the addon. The swapfile itself and the permissions survive the restarts.

##### Edit: Confirmed working solution:

  • install the addon via the repository
  • disable "protection mode" in the addons' settings (always be careful with your new powers!)
  • issue the first three commands inside terminal with changed folders:
fallocate -l 1G /backup/swapfile.swap
mkswap /backup/swapfile.swap
chmod 0600 /backup/swapfile.swap
  • set the third command as init_command in the addons' configuration: swapon /backup/swapfile.swap
Edit: Original post (not optimal, due to increased snapshot size (as described in the next post ))
  • install the adon via [the repository] (https://github.com/hassio-addons/addon-ssh)
  • issue the first three commands inside terminal with changed folders:
fallocate -l 1G /config/www/swapfile.swap
mkswap /config/www/swapfile.swap
chmod 0600 /config/www/swapfile.swap
  • set the third command as init_command in the addons' configuration: swapon /config/www/swapfile.swap

capstan1 avatar Dec 31 '20 14:12 capstan1

@capstan1 it makes doing HA backups a pain... now there's a huge file rather than a few megs. Can the file reside in any other location outside of HA config folder?

JZ-SmartThings avatar Dec 31 '20 23:12 JZ-SmartThings

I found a way to make this "persistant" using the SSH & Web Terminal Addon. When disabling protection mode for the addon you can simply place the swape file inside the Core folders (for example "/config" oder "/config/www") and issue the swapon command as initial command of the addon. The swapfile itself and the permissions survive the restarts.

  • install the adon via [the repository] (https://github.com/hassio-addons/addon-ssh)
  • issue the first three commands inside terminal with changed folders:
fallocate -l 1G /config/www/swapfile.swap
mkswap /config/www/swapfile.swap
chmod 0600 /config/www/swapfile.swap
  • set the third command as init_command in the addons' configuration: swapon /mnt/data/test.swap

HI to all! wonderful .. you are great! i had... an rsp3 + with 1G ram and 200mb of swap file but now.... the swap file is 2Gbit .. I read the post from JZ-SmartThings .... but I thought .... the folder / backup is not part of the backup and I tried to create the swap file in backup folder and i used SSH & Web Terminal Addon like @capstan1 GREAT!!! after a snapshot whith Home Assistant Google Drive Backup and a restart YESSS!!! IT WORKS !!! GREAT! image Cattura

Bellavista avatar Jan 16 '21 00:01 Bellavista

@Bellavista notice that your init_command is not using the same file/location as you have on the swapon command.

We seem to be getting lucky with how swap files are considered/packed by TAR. Notice that 1GB is down to 1MB when compressed:

image

Seems like that 1MB of backup space is indeed a small price to pay for larger swap. However, after looking at init_commands, it made me wonder why not use that to run all of the necessary commands. Something worth trying... thanks for the more detailed steps, the INIT_COMMANDS piece was key for me.

JZ-SmartThings avatar Jan 16 '21 02:01 JZ-SmartThings

Hi, indeed, the snapshot size increases when putting the swapfile inside a folder that gets snapshoted. @JZ-SmartThings: This 1 MB is probably close after restarting. As I understand this, the 1GB is reserved space, hence a compressed file is small. But as swap fills up with time, the snapshots also increase in size. My snapshots grew from around 70MB so 160-200Mb during 1 week of uptime.

I changed the location to the /media folder and excluded this folder from my snapshots (very simple with the Google Drive Addon). Then the snapshots' size don't increase. As @Bellavista mentioned, the /backup folder should be a good choice, too!

However, after looking at init_commands, it made me wonder why not use that to run all of the necessary commands. Something worth trying...

Surely, you could put all commands inside the init_commands. But as the swapfile itself is persistent throughout restarts, it's just not necessary. Don't know if it adds a lot of workload to a restart, though. Only advantage I can see: it's also executed after restoring a snapshot on a fresh install. So you won't need to SSH into the host once after restoring.

capstan1 avatar Jan 16 '21 11:01 capstan1

@capstan1 and @Bellavista I'm so glad you both posted. It all led me to the final ONE-LINER resolution :) which is ultimately the best and most maintainable approach. Basically use the INIT_COMMANDS to run an if statement in the shell and you never really need to visit that shell yourself to make this work. It survives reboots and doesn't make my backups larger because I simply store it in the /backup folder anyway like suggested. It has a bit logic which will re-create the file if it's not there already, etc.

BTW @capstan1 that makes perfect sense... swap file will be all zeros until it's actually used. Now that the uptime is longer it is indeed impacting the backup size like I initially suspected.

Thanks again to you both! Here's the one-line solution which belongs on the Configuration page of the SSH & Web Terminal add-on in HA.

init_commands:
  - if [ ! -f /backup/_swap.swap ]; then fallocate -l 2G /backup/_swap.swap && mkswap /backup/_swap.swap && chmod 0600 /backup/_swap.swap && swapon /backup/_swap.swap ; else swapon /backup/_swap.swap; fi

JZ-SmartThings avatar Jan 19 '21 02:01 JZ-SmartThings

I am not sure why we have to act as an underground/resistance movement but You did it well;) Thank You.

QbaF avatar Jan 19 '21 21:01 QbaF

@QbaF underground teamwork indeed! We also owe some gratitude to @agners who came with the right ammunition!

JZ-SmartThings avatar Jan 19 '21 21:01 JZ-SmartThings

Great job from everyone. I am one of those users who knows little ... indeed nothing ... about programming and linux or yamal commands, I only have a scholastic knowledge ... (but I'm not young ... ;-) internet has been a turn fatal! Well! The effect of the 2GB swap gave the system a lot of stability! Maybe 2 gb is too much I based on the double of system memory rule .... Still great result !!! Thank you all ps. Thank you @agners

Bellavista avatar Jan 19 '21 22:01 Bellavista

Please see the on-going thread here: https://community.home-assistant.io/t/how-to-increase-the-swap-file-size-on-home-assistant-os/272226

I modified the command in the 1st post to have a bit more logic where it checks if swap is already on with dmesg and if so NOT to repeat the command. Otherwise the SSH add-on fails a restart.

Pasting it one last time here and will continue with the community post:

init_commands:
  - if [ ! -f /backup/_swap.swap ]; then fallocate -l 2G /backup/_swap.swap && mkswap /backup/_swap.swap && chmod 0600 /backup/_swap.swap && swapon /backup/_swap.swap ; elif [[ ! $(dmesg|grep _swap.swap) = *swap\ on* ]]; then swapon /backup/_swap.swap; fi

JZ-SmartThings avatar Jan 28 '21 01:01 JZ-SmartThings

Thank you guys, it helped me a lot. I've a raspberry pi 3b and HASSIO was unstable and unavailable almost one time per day ...

johnsmithmad avatar Mar 19 '21 19:03 johnsmithmad

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Jun 11 '21 02:06 stale[bot]

I can't understand it. how to do it: disable "protection mode" in the addon? Explain step by step.

PS Home Assistant OS 6.2 supervisor-2021.09.0 core-2021.9.6

agamemnen avatar Sep 16 '21 19:09 agamemnen

how to do it: disable "protection mode" in the addon?

I will answer my own question. Maybe it will be useful to someone and will save time.

There are several addons. We need exactly:SSH & Web Terminal!

It just has a separate setting: disable "protection mode".

After its activation and if you register the necessary parameters in the config, then everything works!

agamemnen avatar Sep 16 '21 19:09 agamemnen

great solution, thanks I tried to do a simular thing using shell_command, but I kept running into permission issues.

bkbartk avatar Sep 28 '21 13:09 bkbartk

This is all very nice, but I would still prefer a native solution that does not require us to install an unofficial addon. Looking through the documentation, it turns out that this OS isn't completely read-only. We can define network configs, kernel modules and more importantly, udev rules! And all those are persistent!

So here is what I did:

  • After I moved my data disk to an USB SSD, I turned off my rpi3 and took the SSD to another PC
  • There is a single ext-4 partition labelled "hassos-data" on it that was created by the migration procedure
  • I shrank the ext4 partition and created a new swap partition at the end of it
  • Plugged the SSD back into the rpi3 and booted up HASS again
  • Logged into SSH (-p 22222)
  • lsblk now shows "hassos-data" as /dev/sda1 and my swap partition as /dev/sda2
  • I created a new file /etc/udev/rules.d/99-swap.rules on the Host OS with these contents:
KERNEL=="sd?", SUBSYSTEM=="block", DEVPATH=="*usb1/1-1*", NAME="sda", RUN+="/usr/sbin/swapon /dev/sda2"
  • udevadm control --reload-rules && udevadm trigger to see the effects immediately
  • Rebooted to verify that it actually works: It does! And yes, the /etc/udev/rules.d/99-swap.rules file is still present

This is of course a bit shaky since it relies on the block device being added as /dev/sda. I am absolutely not an expert, so this udev rule can certainly be written in a much more robust way. I hope this helps anyone else looking for a solution.

PS: Perhaps it can be improved by introducing partition labels. Maybe the devs would be open to automatically swapon any partition labelled hassos-swap? Just as an idea.

Biont avatar Oct 13 '21 20:10 Biont

I get the output swapon: /backup/_swap.swap: Operation not permitted when trying to execute the command swapon /backup/_swap.swap, anyone knows why?

Vinz87 avatar Nov 02 '21 22:11 Vinz87