RFC: Mount Manager

[pvizeli]

Context

We should not relay on external mounts but in some cases it could be nice to have it like for backups or media.

Decision

We add udisks as a requirement for OS and find a suitable way to handle external mounts.

[ukrolelo]

I don't know why denilsonsa post is market as spam. Just for future reference: @denilsonsa Denilson Sá Maia

Use-cases for NAS mount (SMB/CIFS or NFS) on Home Assistant OS:

• Having an external Media source. (After all, a local /media directory is not good enough for people running the OS on SD card on Raspberry Pi or Odroid.)
• Having a way to store stuff from downloader. (e.g. Saving the videos captured by your Ring Door Bell, which is an example in the official HA documentation)
• Having a proper location for NVR tools (Network Video Recorder), such as Frigate.
• Having MariaDB, InfluxDB (or Graphite, or Prometheus) running inside HA OS, but storing the long-term data onto a NAS.
• Having all these large files outside the HA snapshots (backups), which will help making the snapshots much smaller and faster. (see this issue in influxdb add-on)
• Having fewer writes to the SD card can make it last longer, offloading the writes to a dedicated NAS system that is optimized for storage. There are two main needs:
• Mount stuff to be available to the Home Assistant Core instance (running inside HA OS).
• Mount stuff to be available to Add-ons (also running inside HA OS).

There is also the problem with the credentials. Where should those be stored? I trust HA developers can figure out a good solution.

[pvizeli]

We need someone who to have the time to implement udisk dbus support to the supervisor and future a concept of how the mount points can manage without break functionality and platform independence. If you are a python developer and will doing it, then I will leaf your comments, otherwise, it gets removed until someone comes who start to implement this and need this issue for useful communication and track the work

[ukrolelo]

Hi guyz, so now i have 158 votes, is it enough to speed up this process? https://community.home-assistant.io/t/nas-mount-on-hass-os/320218 As @ludeeus mention here https://github.com/home-assistant/operating-system/pull/1457#issuecomment-878579110 when i had 12votes its not worth it, now its much more. Can somebody from HomeAssistant staff maybe cooperate,please advise, thank you

[frenck]

@ukrolelo Feel free to help out on the development, that is how open source works :)

[ukrolelo]

@frenck xD <3, i thought that feature request is enough?! i don't have knowledge to do this kind of stuff. My advice was just to remove the fstab file so it will not change the existing file that i created with my tutorial for temporary workaround ( https://community.home-assistant.io/t/solved-hassos-mount-nas-network-share/303292 ) and everything will be fine, but i was told its not correct.

[frenck]

i thought that feature request is enough?!

Hehe it won't built itself :)

[alekslyse]

Could it not be solved by adding nfs4 drivers, probably cifs also and let us mount in fstab through the usb? The usb is a ok way to override and persist changes on my own risk

[pvizeli]

No, only implementation of this RFC will get approved. We will not more support any kind of hacks into OS or supervisor

[raman325]

@pvizeli thank you for the references on Discord. Here's what I have discovered so far, posting here so we can validate what's needed:

• Add new module for udisks2
• Add GetBlockDevices method and SupportedFilesystems property
• Add class for org.freedesktop.UDisks2.Block and org.freedesktop.UDisks2.Filesystem
  • To read from fstab, add support for org.freedesktop.UDisks2.Block.Configuration property
  • To update fstab, add support for org.freedesktop.UDisks2.Block.AddConfigurationItem and org.freedesktop.UDisks2.Block.RemoveConfigurationItem methods - maybe add org.freedesktop.UDisks2.Block.UpdateConfigurationItem method for convenience
  • To mount and unmount filesystems, add support for org.freedesktop.UDisks2.Filesystem.Mount and org.freedesktop.UDisks2.Filesystem.Unmount methods
  • To provide info about the filesystem, add org.freedesktop.UDisks2.Filesystem.Mountpoints, org.freedesktop.UDisks2.Filesystem.Size, org.freedesktop.UDisks2.Block.Device, org.freedesktop.UDisks2.Block.Id, and org.freedesktop.UDisks2.Block.ReadOnly properties

Let me know what you think about the proposed scope of implementation, thanks!

[raman325]

not sure if I did this right but I put up a PR based on what I've learned. Let me know if I am way off the mark!

[agners]

Looking at it quickly seems to me that this implements the UDisks2 D-Bus interfaces we need, so definitely a good start :+1:

I know that @mdegat01 worked on D-Bus a bit, also exploring the use of dbus-next proxy objects. With using that functionality this might become easier actually.

One thing I noticed is that UDisks2 relies on regular /etc/fstab under the hood, and writes to it. I expect that today the org.freedesktop.UDisks2.Block.Add/Update/RemoveConfigurationItem will fail on HAOS because UDisks2 can't write to /etc/fstab. So we'll have to work on that on the OS side as well.

[agners]

Btw, looking at Gnome Disk Manager is a good way to explore the capabilities of UDisks2, as its using that under the hood. Configuring a new block device via its interface looks as follows:

It actually seems that CIFS and NFS is not among the supported file systems for UDisks2. At least through the UI and udisksctl it seems not to be possible to mount CIFS or NFS storage. Reading this StackExchange thread it seems that is not supported by UDisks2....

Since UDisks2 only seems to support block devices, and manages /etc/fstab instead of systemd mount targets (which is how OS internal mounts are managed in HAOS), I wonder if using UDisks2 is the right appraoch here.

Maybe creating systemd units is the way to go. There is no D-Bus API for that, but we could define a dedicated directory where we store the mount units and bind mount it for the Supervisor. :thinking:

[raman325]

Reading this StackExchange

Thanks for the references. I too struggled to find the information asked in the StackExchange but the lack of information made me worry that this was indeed the case, so thank you for confirming. Does that negate the need for udisks2 support, or would we benefit from it anyway?

Regarding systemd units, I will admit that my knowledge of Linux and D-Bus is limited, I only took this on because I was able to use the existing D-Bus APIs in the supervisor code to understand how to connect the dots. This may require someone else to carry it any further.

[pvizeli]

I guess we can make it working with udisk2 and hopefully udisk2 get future developed or later replaced with an dbus systemd things. Manage files over a bind mount is to hacky

[raman325]

can one of you help me figure out how to generate fixtures for udisks2? happy to write tests for the above but I am not familiar enough with D-Bus to query the data

[agners]

I usually use busctl to poke around. E.g. to get a tree of objects available

busctl tree org.freedesktop.UDisks2

To get properties/methods of an object

busctl introspect org.freedesktop.UDisks2 /org/freedesktop/UDisks2/drives/Generic_MassStorageClass_000000001209

You can then issue calls, and use the -j parameter to generate JSON. This should give you useful data for test fixtures.

[agners]

I guess we can make it working with udisk2 and hopefully udisk2 get future developed or later replaced with an dbus systemd things. Manage files over a bind mount is to hacky

I don't expect that UDisks adds these features. it seems that the scope of UDisks is really to deal with physical/local storage:

The UDisks project provides a daemon, tools and libraries to access and manipulate disks, storage devices and technologies

Afaik, UDisks is mainly driven from needs of Gnome, and mounting CIFS/NFS stores is handled by gvfs in that world. UDisks is really mainly about managing disks, mounting them is kinda a side feature, it seems...

Manage files over a bind mount is to hacky

In the end it needs to written to a file at one point :man_shrugging: At least systemd mount files are well defined. Essentially, it would make Supervisor dependent on systemd to provide mount capabilities, which I feel is a reasonable dependency.

If we want to keep the file format abstracted from Supervisor, then I guess we'd need to add a mount manager feature to OS Agent, which reads/writes these mounts into systemd units...

[pvizeli]

Well that have the same drawback, we have to open the hole systemd units and people can start to overwrite things. Enable fdisk file would still open it up for some use case but not that aggressive as open the full systemd units for overwrittes

[agners]

Well that have the same drawback, we have to open the hole systemd units and people can start to overwrite things.

Those would be mount units, and Supervisor would only write the specifically allowed properties etc, so how could people overwrite things?

In the end, we trust Supervisor already anyways, it has access to Docker socket (which essentially allows to do anything on the host through starting privileged containers & bind mounts... e.g. write arbitrary unit files to /run/systemd/system, which is picked up by systemd).

Allowing mounts through fstab can likely also exploited to mount units directly into a path parsed by systemd...

[dianlight]

I'm joining this thread and hope my 2 cents can help.

Background/As-Is

As author of the SambaNAS addon, I currently use a very dirty method (addon to host via ssh connection) to manage the mount/umount of disks connected to hassos. Currently "SambaNAS" via SSH gives the command to mount disks on the host directly in media; in this way all dockers ( supervisor and core included ) see the subdirectories with the contents of the disks inside the media directory after a restart¹ or after an explicit mount inside the running docker image itself.

Migrate to UDISK2 and UDEV

I did some tests using the DBUS/UDISK to directly command the mount/umount of the disks from within the addon but to replace the "rough" method used in SambaNAS there are some fundamental pieces missing. I think the same pieces can be useful to be able to extend the system to "non block devices" such as NFS or CIFS currently not supported by udisk2

Problems and solutions (in order of importance as I see it)

• [ ] Udisk2 mount path. Udisk2 basically manages 2 paths on which to mount the disks. /run/media/<user>/* or /media:

  • How to show mounted disks to supervisor/core/addons? The simplest thing would be to be able to mount the udisk media directory at the start of the docker images. As done with /run/udev or whatever but in rw since the disks must be usable by all addons.
  • How to make a docker image able to 'read/write' a disk that has been mounted after the image has been run? If I mount a disk inside a path visible from a running docker I don't see the new disk until I reboot the image. I tried to search for a solution but I couldn't even find any reference to why it happens.

• [x] Control the mount/umount block devices from core/supervisor/addon; the DBUS interface of Udisk2 is quite complete having been created for a desktop UI control so create in ha/setting a panel dedicated to the control of external disks which communicates via DBUS with the host.

• [ ] Control the mount/umount non block devices (like CIFS NFS etc). Creating a cifs/nfs udisk2 module seems like a viable option based on how the lsm module works. Obviously I don't think it's an interesting CR for udisk2 so it would lead to using a fork for the hassos images. An alternative could be the creation of a new project 'udiskN' which can mount and manage the network drivers via dbus in a very similar way to udisk2

[Drizzt321]

While it might be a little inconvenient, I don't see a problem with requiring a docker container restart to add an extra disk. Seems like a small price to pay for this functionality.

[austwhite]

While it might be a little inconvenient, I don't see a problem with requiring a docker container restart to add an extra disk. Seems like a small price to pay for this functionality.

While restarting a docker container may be fine for power users and the like, also remember Home Assistant is being aimed at a simpler market these days also. With the Blue and then Yellow, anything done needs to cater for the appliance like nature of those devices, and for the set and forget mentality newer users of HAOS may have. Convenience is becoming more important. I suppose it could be a case of shutdown the whole HA Hub, plug in a drive, boot it back up, but that would only be a band-aid solution.
While not ideal, as plug and play would be the most acceptable, I like the fact things are moving forward.

[cju-mt]

I would love to see a mount manager added to HA. I am not a power user yet but I love the simplicity of HA and with all the addons that are available it only makes sense to have external storage. I use the Frigate NVR addon and I know it is going to quickly fill up my VM. I will have to keep my eye on it. I can tell you though, I am happy to restart the docker container. It would seem to me that anyone that is using HA is in some manner a little techy :)

[Drizzt321]

RE: restarting a docker container...well, why not just have to restart HA docker host system itself? Annoying, but if it's rare, I don't see users objecting to something like that for the extra functionality.

[austwhite]

@Drizzt321 Could work if it was a simple button to restart within an integration or add-on. Anything that is done would need to follow the Home Assistant roadmap of doing everything from the UI and being simple for end users......... If it is a once off when storage is added.... Well, how often do people add storage devices? It would just need to be done in a simple way that any half trained monkey could do.

[cju-mt]

Just trying to keep the momentum going since this was opened 2-13-2021. I wish I was proficient enough to help. I am certainly glad to help in any way I can and to be a beta tester. I just can't say enough good things about home assistant. One Home Automation System to Rule them all.

[ukrolelo]

@pvizeli What is that, is it ready for testing? Will it be able to mount remote smb share? :>

[agners]

This got closed by accident. Since #3848 is just a building block for the Mount Manager this issue should remain open.

[PeteDenmark]

I am a total newbie when it comes to Home Assistant, but just want to second the need for a way to mount extra drives.

Right now I'm using udev rules and SSH access to port 22222 to mount both usb drives and my NAS. Well - not a "real" NAS, as it is just a harddisk attached to my router.

As a beginner, i am quite sure that during my attempts to find a way to mount drives I have exposed my system to many security problems, without knowing it, because mounting extra drives has been such an uphill battle, with no official support or guidance.

Having an addon that actually could do this, even if i required a restart every time a drive needed to be mounted, would have been so helpful, and a lot less stressful.

I have zero experience with Linux, but I'll gladly help to test things out, if input from a beginner is any help at all.

[DennisFury]

Home assistant needed this years ago. Dev gods please help, I'll donate some beer money to the cause