Unable to start endlessh on port 22

[nicoonoclaste]

trafficstars

When trying to start endlessh on port 22, on a machine using systemd, the service fails to start due to getting EACCESS on the bind(2) call to open the socket.

As endlessh only reports fatal: Permission denied, I took a recording of the process' activity under strace, running as the systemd service:

Jan 30 22:19:10 neon.citronna.de systemd[1]: Started Endlessh SSH Tarpit.
Jan 30 22:19:10 neon.citronna.de strace[2060944]: execve("/usr/bin/endlessh", ["/usr/bin/endlessh", "-p22"], 0x7ffe5c745d68 /* 5 vars */) = 0
Jan 30 22:19:10 neon.citronna.de strace[2060944]: brk(NULL)                               = 0x55809d08c000
[loading libraries...]
Jan 30 22:19:10 neon.citronna.de strace[2060944]: openat(AT_FDCWD, "/etc/endlessh/config", O_RDONLY) = -1 ENOENT (No such file or directory)
Jan 30 22:19:10 neon.citronna.de strace[2060944]: rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=SA_RESTORER|SA_INTERRUPT|SA_NODEFER|SA_RESETHAND, sa_restorer=0x7f5f98187100}, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=0}, 8) = 0
Jan 30 22:19:10 neon.citronna.de strace[2060944]: rt_sigaction(SIGTERM, {sa_handler=0x55809b5eddd0, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7f5f98187100}, NULL, 8) = 0
Jan 30 22:19:10 neon.citronna.de strace[2060944]: rt_sigaction(SIGHUP, {sa_handler=0x55809b5edde0, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7f5f98187100}, NULL, 8) = 0
Jan 30 22:19:10 neon.citronna.de strace[2060944]: rt_sigaction(SIGUSR1, {sa_handler=0x55809b5eddf0, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7f5f98187100}, NULL, 8) = 0

[we are now in server_create]
Jan 30 22:19:10 neon.citronna.de strace[2060944]: socket(AF_INET6, SOCK_STREAM, IPPROTO_IP) = 3
Jan 30 22:19:10 neon.citronna.de strace[2060944]: setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
Jan 30 22:19:10 neon.citronna.de strace[2060944]: setsockopt(3, SOL_IPV6, IPV6_V6ONLY, [0], 4) = 0
Jan 30 22:19:10 neon.citronna.de strace[2060944]: bind(3, {sa_family=AF_INET6, sin6_port=htons(22), inet_pton(AF_INET6, "::", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = -1 EACCES (Permission denied)

[BOOM!]
Jan 30 22:19:10 neon.citronna.de strace[2060944]: write(2, "endlessh: fatal: Permission deni"..., 35
Jan 30 22:19:10 neon.citronna.de systemd[1]: endlessh.service: Main process exited, code=exited, status=1/FAILURE
Jan 30 22:19:10 neon.citronna.de strace[2060949]: write(2, "endlessh: fatal: Permissi
Jan 30 22:19:10 neon.citronna.de strace[2060944]: write(2, "endlessh: fatal: Permissi
Jan 30 22:19:10 neon.citronna.de strace[2060944]:  = 35
Jan 30 22:19:10 neon.citronna.de systemd[1]: endlessh.service: Failed with result 'exit-code'.
Jan 30 22:19:10 neon.citronna.de strace[2060944]: exit_group(1)
Jan 30 22:19:28 neon.citronna.de systemd[1]: Stopped Endlessh SSH Tarpit.

I was unable to find immediately what causes this, though I expect it is related to the security-related settings in endless.service.

I didn't search very hard, though, as I have a more satisfying solution via socket activation...

[skeeto]

I have little idea how the systemd stuff works, and I haven't personally tested util/endless.service, so I can't say what would cause this error. There's a comment about binding privileged ports in the service file. Socket activation, per the PR, would be nice if systemd is flexible enough about it.

[smoeding]

Could this be caused by SELinux being active? Maybe /usr/bin/endlessh needs the same SELinux security context as /usr/sbin/sshd?

[0xDedSec42]

I think I found the reason but do not have a solution. I'm using Ubuntu thus there is no SELinux and the systemd service still won't start on any port < 1024.

systemctl edit endlessh.service

And adding

[Service]
AmbientCapabilities=CAP_NET_BIND_SERVICE

should do the trick but doesn't.

[0xDedSec42]

After doing

echo 'net.ipv4.ip_unprivileged_port_start=0' > /etc/sysctl.d/50-unprivileged-ports.conf
sysctl --system

and rebooting my system EndleSSH starts on boot on port 22.

[Directory]

I think I solved this. You completely overlooked the blatantly commented "documentation" in the service file to begin with. Lets take a look at it again. Around the middle there are these lines.

## If you want Endlessh to bind on ports < 1024
## 1) run:
##     setcap 'cap_net_bind_service=+ep' /usr/local/bin/endlessh
## 2) uncomment following line
#AmbientCapabilities=CAP_NET_BIND_SERVICE
## 3) comment following line
PrivateUsers=true

Well it blatantly tells us what do do right here. Edit the service with sudo nano /etc/systemd/system/endlessh.service. Uncomment the AmbientCapabilities line. Comment out the PrivateUsers line, save the file, and run setcap 'cap_net_bind_service=+ep' /usr/local/bin/endlessh. Now we can set Port 22 in the sudo nano /etc/endlessh/config. Then restart the service sudo systemctl daemon-reload && sudo systemctl restart endlessh.service. Now when checking the status with sudo systemctl status endlessh.service it shows its running fine.

[ekzyis]

Can confirm, it worked using the instructions in the service file. Also didn't look into the service file :see_no_evil:

[ghost]

I can confirm that the following comment by @Directory solved endlessh setup for me after installing the debian package.

That said, it would be nice if the default endlessh port was 22, not really much point in setting it as a non-default port IMHO. If this package could become a standard server security hardening method I think this could really serve as a deterrent to casual port scanning.

I think I solved this. You completely overlooked the blatantly commented "documentation" in the service file to begin with. Lets take a look at it again. Around the middle there are these lines.

## If you want Endlessh to bind on ports < 1024
## 1) run:
##     setcap 'cap_net_bind_service=+ep' /usr/local/bin/endlessh
## 2) uncomment following line
#AmbientCapabilities=CAP_NET_BIND_SERVICE
## 3) comment following line
PrivateUsers=true

Well it blatantly tells us what do do right here. Edit the service with sudo nano /etc/systemd/system/endlessh.service. Uncomment the AmbientCapabilities line. Comment out the PrivateUsers line, save the file, and run setcap 'cap_net_bind_service=+ep' /usr/local/bin/endlessh. Now we can set Port 22 in the sudo nano /etc/endlessh/config. Then restart the service sudo systemctl daemon-reload && sudo systemctl restart endlessh.service. Now when checking the status with sudo systemctl status endlessh.service it shows its running fine.

[trallnag]

What is missing in the docs that the daemon needs to be reloaded

[Pickled-Aries-75]

I get this error " Failed to set capabilities on file `/usr/local/bin/endlessh' (Invalid argument) The value of the capability argument is not permitted for a file. Or the file is not a regular (non-symlink) file" ubuntu server 20.04

[jpettitt]

@Pickled-Aries-75 did you remember to run cap_net_bind_service=+ep' /usr/local/bin/endlessh - It's working on Ubuntu LTS for me following the doc above

[Mist-Hunter]

For anyone else running Debian wondering how to apply the directions above, I got it working after changing a few paths.

setcap 'cap_net_bind_service=+ep' /usr/bin/endlessh
sed -i 's|#AmbientCapabilities|AmbientCapabilities|g' /lib/systemd/system/endlessh.service
sed -i 's|PrivateUsers|#PrivateUsers|g' /lib/systemd/system/endlessh.service
systemctl daemon-reload
systemctl restart endlessh.service
systemctl status endlessh.service

The service unit location was throwing me off for a bit, but the changes above resulted in:

> systemctl status endlessh.service
● endlessh.service - Endlessh SSH Tarpit
     Loaded: loaded (/lib/systemd/system/endlessh.service; disabled; vendor preset: enabled)
     Active: active (running) since Sat 2022-04-02 16:32:39 PDT; 5s ago
       Docs: man:endlessh(1)
   Main PID: 2013 (endlessh)
      Tasks: 1 (limit: 2340)
     Memory: 184.0K
        CPU: 25ms
     CGroup: /system.slice/endlessh.service
             └─2013 /usr/bin/endlessh

Apr 02 16:32:39 - systemd[1]: Started Endlessh SSH Tarpit.

[SpareSimian]

@Mist-Hunter , don't edit the file in /lib. That file can be overwritten on package updates. Either use "systemctl edit" or copy the systemd unit file to /etc/systemd/system and edit the copy. Using systemctl edit you can create a "drop-in" file in /etc that overrides just the settings you want to change, so that changes in the distribution unit file will be picked up when you update and your overrides will be applied to the new unit file. https://flatcar-linux.org/docs/latest/setup/systemd/drop-in-units/

[Frew777]

By this instruction I make it works on Debian 11 on 22 port

1. Run this command setcap 'cap_net_bind_service=+ep' /usr/bin/endlessh
2. Edit service systemctl edit endlessh.service Add these strings after ### Anything between here and the comment below will become the new contents of the file

[Service]
AmbientCapabilities=CAP_NET_BIND_SERVICE
PrivateUsers=false

Like this

### Editing /etc/systemd/system/endlessh.service.d/override.conf
### Anything between here and the comment below will become the new contents of the file

[Service]
AmbientCapabilities=CAP_NET_BIND_SERVICE
PrivateUsers=false

### Lines below this comment will be discarded

3. Reload config systemctl daemon-reload
4. Finally trying to restart the service systemctl restart endlessh.service && systemctl status endlessh.service

[mattfbacon]

I found that I didn't need to run the setcap command to fix the issue. It seems a bit dirty and I'm not sure if it would also be overridden on package updates.