gluster
allow mounting through unix socket via fstab
Description of problem:
I am reviving the ticket https://github.com/gluster/glusterfs/issues/331 which was raised on 2017.
Gluster is simple to install, and in a few cases I am installing Gluster storage dedicated to a single application.
I am using 127.0.0.1 in fstab to mount the filesystem, but I'd like to use /var/run/glusterd.socket
The exact command to reproduce the issue:
being able to use the following inside fstab:
/var/run/glusterd.socket:forgejo /test glusterfs defaults,_netdev,negative-timeout=10,attribute-timeout=30,fopen-keep-cache,direct-io-mode=enable,transport=unix 0 0
And, as specified in the issue #331 it throws an error like this:
[2024-10-03 08:11:15.029036 +0000] E [glusterfsd-mgmt.c:2371:mgmt_getspec_cbk] 0-mgmt: failed to fetch volume file (key:forgejo.unix)
It seems that it's appending .unix, and if that's really the problem, it should be fairly easy to fix it and I see no reason the push it back.
The full output of the command that failed:
[2024-10-03 08:11:15.019238 +0000] I [MSGID: 100030] [glusterfsd.c:2767:main] 0-/usr/sbin/glusterfs: Started running version [{arg=/usr/sbin/glusterfs}, {version=10.1}, {cmdlinestr=/usr/sbin/glusterfs --fopen-keep-cache=true --direct-io-mode=enable --attribute-timeout=30 --negative-timeout=10 --process-name fuse --volfile-server=/var/run/glusterd.socket --volfile-server-transport=unix --volfile-id=/forgejo.unix /test}]
[2024-10-03 08:11:15.023844 +0000] I [glusterfsd.c:2447:daemonize] 0-glusterfs: Pid of current running process is 1139457
[2024-10-03 08:11:15.028742 +0000] I [MSGID: 101190] [event-epoll.c:667:event_dispatch_epoll_worker] 0-epoll: Started thread with index [{index=0}]
[2024-10-03 08:11:15.028864 +0000] I [MSGID: 101190] [event-epoll.c:667:event_dispatch_epoll_worker] 0-epoll: Started thread with index [{index=1}]
[2024-10-03 08:11:15.029023 +0000] E [glusterfsd-mgmt.c:2170:mgmt_getspec_cbk] 0-glusterfs: failed to get the 'volume file' from server
[2024-10-03 08:11:15.029036 +0000] E [glusterfsd-mgmt.c:2371:mgmt_getspec_cbk] 0-mgmt: failed to fetch volume file (key:/forgejo.unix)
[2024-10-03 08:11:15.029297 +0000] W [glusterfsd.c:1458:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libgfrpc.so.0(+0x10510) [0x7f7c0c8d0510] -->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x747) [0x563699070eb7] -->/usr/sbin/glusterfs(cleanup_and_exit+0x5b) [0x5636990680bb] ) 0-: received signum (0), shutting down
[2024-10-03 08:11:15.029339 +0000] I [fuse-bridge.c:7057:fini] 0-fuse: Unmounting '/test'.
[2024-10-03 08:11:15.031877 +0000] I [fuse-bridge.c:7061:fini] 0-fuse: Closing fuse connection to '/test'.[2024-10-03 08:11:15.019238 +0000] I [MSGID: 100030] [glusterfsd.c:2767:main] 0-/usr/sbin/glusterfs: Started running version [{arg=/usr/sbin/glusterfs}, {version=10.1}, {cmdlinestr=/usr/sbin/glusterfs --fopen-keep-cache=true --direct-io-mode=enable --attribute-timeout=30 --negative-timeout=10 --process-name fuse --volfile-server=/var/run/glusterd.socket --volfile-server-transport=unix --volfile-id=/forgejo.unix /test}]
[2024-10-03 08:11:15.023844 +0000] I [glusterfsd.c:2447:daemonize] 0-glusterfs: Pid of current running process is 1139457
[2024-10-03 08:11:15.028742 +0000] I [MSGID: 101190] [event-epoll.c:667:event_dispatch_epoll_worker] 0-epoll: Started thread with index [{index=0}]
[2024-10-03 08:11:15.028864 +0000] I [MSGID: 101190] [event-epoll.c:667:event_dispatch_epoll_worker] 0-epoll: Started thread with index [{index=1}]
[2024-10-03 08:11:15.029023 +0000] E [glusterfsd-mgmt.c:2170:mgmt_getspec_cbk] 0-glusterfs: failed to get the 'volume file' from server
[2024-10-03 08:11:15.029036 +0000] E [glusterfsd-mgmt.c:2371:mgmt_getspec_cbk] 0-mgmt: failed to fetch volume file (key:/forgejo.unix)
[2024-10-03 08:11:15.029297 +0000] W [glusterfsd.c:1458:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libgfrpc.so.0(+0x10510) [0x7f7c0c8d0510] -->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x747) [0x563699070eb7] -->/usr/sbin/glusterfs(cleanup_and_exit+0x5b) [0x5636990680bb] ) 0-: received signum (0), shutting down
[2024-10-03 08:11:15.029339 +0000] I [fuse-bridge.c:7057:fini] 0-fuse: Unmounting '/test'.
[2024-10-03 08:11:15.031877 +0000] I [fuse-bridge.c:7061:fini] 0-fuse: Closing fuse connection to '/test'.
Expected results:
Mandatory info:
- The output of the gluster volume info command:
# gluster volume info
Volume Name: forgejo
Type: Replicate
Volume ID: 140a360d-db59-4af0-9f91-b8d2285b238d
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: test-forgejo-witness.example.org:/data/glusterfs/forgejo/brick_forgejo/brick
Brick2: test-forgejo01.example.org:/data/glusterfs/forgejo/brick_forgejo/brick
Brick3: test-forgejo02.example.org:/data/glusterfs/forgejo/brick_forgejo/brick
Options Reconfigured:
performance.client-io-threads: true
nfs.disable: on
transport.address-family: inet
storage.fips-mode-rchecksum: on
cluster.granular-entry-heal: on
client.event-threads: 3
performance.cache-size: 256MB
performance.io-cache: true
performance.io-thread-count: 16
performance.readdir-ahead: true
storage.owner-gid: 130
storage.owner-uid: 130
features.barrier: disable
- The output of the gluster volume status command:
# gluster volume status
Status of volume: forgejo
Gluster process TCP Port RDMA Port Online Pid
------------------------------------------------------------------------------
Brick test-forgejo-witness.example.org:/data/
glusterfs/forgejo/brick_forgejo/brick 53950 0 Y 1811
Brick test-forgejo01.example.org:/data/gluste
rfs/forgejo/brick_forgejo/brick 57698 0 Y 1844
Brick test-forgejo02.example.org:/data/gluste
rfs/forgejo/brick_forgejo/brick 58762 0 Y 1871
Self-heal Daemon on localhost N/A N/A Y 1836
Self-heal Daemon on test-forgejo01.geant.or
g N/A N/A Y 1921
Self-heal Daemon on test-forgejo02.geant.or
g N/A N/A Y 1947
Task Status of Volume forgejo
------------------------------------------------------------------------------
There are no active volume tasks
- The output of the gluster volume heal command:
n/a
**- Provide logs present on following locations of client and server nodes -
/var/log/glusterfs/
**- Is there any crash ? Provide the backtrace and coredump
n/a
Additional info:
- The operating system / glusterfs version:
Ubuntu 22.04 glusterfs-server 10.1-1
@maxadamo I am not understanding the usecase. You want to use unix socket in control path or data path?
@maxadamo I am not understanding the usecase. You want to use unix socket in control path or data path?
@pranithk - I recall that you had a draft implementation of connecting via unix socket and it gave 5% perf. improvement or so - and the motivation was for hyper-converged setups.
yes, hyper-converged setup is the use case. I have a design for a couple of applications with each node connecting to Gluster locally. Since it works from the command line, at the moment I am handling the mountpoint through scripts and systemd.
@pranithk this is an excerpt from the logs:
[2025-07-11 18:43:03.682723 +0000] I [MSGID: 100030] [glusterfsd.c:2767:main] 0-/usr/sbin/glusterfs: Started running version [{arg=/usr/sbin/glusterfs}, {version=10.1}, {cmdlinestr=/usr/sbin/glusterfs --fopen-keep-cache=true --direct-io-mode=enable --attribute-timeout=30 --negative-timeout=10 --process-name fuse --volfile-server=/var/run/glusterd.socket --volfile-server-transport=unix --volfile-id=moodle_www.unix /test}]
and this is the mount point in /etc/fstab:
/var/run/glusterd.socket:moodle_www /test glusterfs defaults,_netdev,negative-timeout=10,attribute-timeout=30,fopen-keep-cache,direct-io-mode=enable,transport=unix 0 0
I defined the volume name moodle_www, but it's trying to mount volfile-id=moodle_www.unix.
It's adding .unix, which needs to be removed.