pytest-testinfra
pytest-testinfra copied to clipboard
pytest-dev
host.socket.get_listening_sockets() does not work on Ubuntu 18.04.1 LTS
I'm using this snippet of code to test listening ports:
def test_listening_socket(host):
listening = host.socket.get_listening_sockets()
for spec in (
"tcp://:::22",
"tcp://:::5432",
):
socket = host.socket(spec)
assert socket.is_listening
but it fails on Ubuntu 18.04.1 LTS:
cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"
==================================================================================================================================== FAILURES =====================================================================================================================================
_______________________________________________________________________________________________________________ test_listening_socket[paramiko://alpha] _______________________________________________________________________________________________________________
host = <testinfra.host.Host object at 0x7fc1b010e190>
def test_listening_socket(host):
> listening = host.socket.get_listening_sockets()
test_alpha.py:80:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python2.7/dist-packages/testinfra/modules/socket.py:171: in get_listening_sockets
for sock in cls(None)._iter_sockets(True):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <socket None://None>, listening = True
def _iter_sockets(self, listening):
cmd = 'ss --numeric'
if listening:
cmd += ' --listening'
else:
cmd += ' --all'
if self.protocol == 'tcp':
cmd += ' --tcp'
elif self.protocol == 'udp':
cmd += ' --udp'
elif self.protocol == 'unix':
cmd += ' --unix'
for line in self.run(cmd).stdout_bytes.splitlines()[1:]:
# Ignore unix datagram sockets.
if line.split(None, 1)[0] == b'u_dgr':
continue
splitted = line.decode().split()
# If listing only TCP or UDP sockets, output has 5 columns:
# (State, Recv-Q, Send-Q, Local Address:Port, Peer Address:Port)
if self.protocol in ('tcp', 'udp'):
protocol = self.protocol
status, local, remote = (
splitted[0], splitted[3], splitted[4])
# If listing all or just unix sockets, output has 6 columns:
# Netid, State, Recv-Q, Send-Q, LocalAddress:Port, PeerAddress:Port
else:
protocol, status, local, remote = (
> splitted[0], splitted[1], splitted[4], splitted[5])
E IndexError: list index out of range
/usr/local/lib/python2.7/dist-packages/testinfra/modules/socket.py:238: IndexError
Can you please fix your code formatting, I find this example hard to follow due to the weird formatting it's currently in.
Please see https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet
Also, you didn't state which version of testinfra you have installed. Please provide this pip freeze -l and copy and paste that into a new comment.
Please also provide an a copy and paste of netstat -ant | grep LIST to prove that those ports are listening.
FWIW - I cannot reproduce your claim that 18.04 the socket module does not work on ubuntu 18.04. From initial tests using the latest release of testinfra==1.14.0 the socket module does indeed work.
Hello, I'm having a similar issue. I can open a new issue if you like. I'm testing in docker, ubuntu 18.04. I have the same test in trusty, and that isn't having any issue.
Tests
def test_http(host):
http = host.socket('tcp://80')
assert http.is_listening
def test_https(host):
https = host.socket('tcp://443')
assert https.is_listening
netstat output:
# netstat -ant | grep LIST
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
Hi @adborden , this issue is talking about get_listening_sockets(), so I'm not sure this is the same issue here, it is ?
While investigating issues on socket module, I found a bug with ss (testinfra use ss or netstat given what's available): https://github.com/philpep/testinfra/issues/422 , looks related to your issue.
Looks like this story hasn't ended yet. We faced similar problem, but with LXD instead of Nginx. Here is how our tests crashed:
< ... SKIPPED ...>
=================================== FAILURES ===================================
________ test_ports[ansible://local-test-42-bionic-default-tcp://8443] _________
host = <testinfra.host.Host object at 0x7faec56facc0>, port = 'tcp://8443'
@pytest.mark.parametrize("port", [
(lxd_listen_tcp),
(lxd_listen_unix),
])
def test_ports(host, port):
"""Testing for listeners on port."""
> assert host.socket(port).is_listening
tests/test_default.py:63:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.5/dist-packages/testinfra/modules/socket.py:112: in is_listening
sockets = list(self._iter_sockets(True))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <socket tcp://8443>, listening = True
def _iter_sockets(self, listening):
cmd = '%s --numeric'
if listening:
cmd += ' --listening'
else:
cmd += ' --all'
if self.protocol == 'tcp':
cmd += ' --tcp'
elif self.protocol == 'udp':
cmd += ' --udp'
elif self.protocol == 'unix':
cmd += ' --unix'
for line in self.run(cmd, self._command).stdout_bytes.splitlines()[1:]:
# Ignore unix datagram sockets.
if line.split(None, 1)[0] == b'u_dgr':
continue
splitted = line.decode().split()
# If listing only TCP or UDP sockets, output has 5 columns:
# (State, Recv-Q, Send-Q, Local Address:Port, Peer Address:Port)
if self.protocol in ('tcp', 'udp'):
protocol = self.protocol
status, local, remote = (
> splitted[0], splitted[3], splitted[4])
E IndexError: list index out of range
/usr/local/lib/python3.5/dist-packages/testinfra/modules/socket.py:239: IndexError
------------------------------ Captured log call -------------------------------
INFO testinfra:base.py:281 RUN CommandResult(command=b"lxc exec local-test-42-bionic-default --mode=non-interactive -- /bin/sh -c 'command -v ss'", exit_status=0, stdout=b'/bin/ss\n', stderr=None)
INFO testinfra:base.py:281 RUN CommandResult(command=b"lxc exec local-test-42-bionic-default --mode=non-interactive -- /bin/sh -c '/bin/ss --numeric --listening --tcp'", exit_status=0, stdout=b'State Recv-Q Send-Q Local Address:Port Peer Address:Port \nLISTEN0 32 10.120.149.1:53 0.0.0.0:* \nLISTEN0 128 127.0.0.53%lo:53 0.0.0.0:* \nLISTEN0 128 0.0.0.0:22 0.0.0.0:* \nLISTEN0 32 [fd42:d8d2:8099:e96e::1]:53 [::]:* \nLISTEN0 32 [fe80::d0d2:7eff:fefd:8273]%lxdbr0:53 [::]:* \nLISTEN0 128 [::]:22 [::]:* \nLISTEN0 128 *:8443 *:* \n', stderr=None)
===================== 1 failed, 17 passed in 9.97 seconds ======================
writing pytestdebug information to /home/user/ansible/lxd/molecule/default/pytestdebug.log
wrote pytestdebug information to /home/user/ansible/lxd/molecule/default/pytestdebug.log
And here is an interesting part. How TCP socket will be checked in case of Ubuntu 16.04/xenial:
root@builder:~# lxc exec local-test-42-xenial-default --mode=non-interactive -- /bin/sh -c '/bin/ss --numeric --listening --tcp'
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 5 10.27.204.1:53 *:*
LISTEN 0 128 *:22 *:*
LISTEN 0 5 fd42:419e:64e2:fe80::1:53 :::*
LISTEN 0 5 fe80::fc16:eaff:fe6e:f9ed%lxdbr0:53 :::*
LISTEN 0 128 :::22 :::*
LISTEN 0 128 :::8443 :::*
And the same for Ubuntu 18.04/bionic (which crashed):
root@builder:~# lxc exec local-test-42-bionic-default --mode=non-interactive -- /bin/sh -c '/bin/ss --numeric --listening --tcp'
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN0 32 10.120.149.1:53 0.0.0.0:*
LISTEN0 128 127.0.0.53%lo:53 0.0.0.0:*
LISTEN0 128 0.0.0.0:22 0.0.0.0:*
LISTEN0 128 127.0.0.1:8443 0.0.0.0:*
LISTEN0 32 [fd42:d8d2:8099:e96e::1]:53 [::]:*
LISTEN0 32 [fe80::d0d2:7eff:fefd:8273]%lxdbr0:53 [::]:*
LISTEN0 128 [::]:22 [::]:*
As you can see there is something strange with first two columns. They just merged to one by some reason. Digging deeper and trying to run same comand within bionic:
root@local-test-42-bionic-default:~# /bin/ss --numeric --listening --tcp
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 32 10.120.149.1:53 0.0.0.0:*
LISTEN 0 128 127.0.0.53%lo:53 0.0.0.0:*
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 128 127.0.0.1:8443 0.0.0.0:*
LISTEN 0 32 [fd42:d8d2:8099:e96e::1]:53 [::]:*
LISTEN 0 32 [fe80::d0d2:7eff:fefd:8273]%lxdbr0:53 [::]:*
LISTEN 0 128 [::]:22 [::]:*
Looks ok, but here is subshell version (same as lxc exec ... works?):
root@local-test-42-bionic-default:~# echo "$(/bin/ss --numeric --listening --tcp)"
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN0 32 10.120.149.1:53 0.0.0.0:*
LISTEN0 128 127.0.0.53%lo:53 0.0.0.0:*
LISTEN0 128 0.0.0.0:22 0.0.0.0:*
LISTEN0 128 127.0.0.1:8443 0.0.0.0:*
LISTEN0 32 [fd42:d8d2:8099:e96e::1]:53 [::]:*
LISTEN0 32 [fe80::d0d2:7eff:fefd:8273]%lxdbr0:53 [::]:*
LISTEN0 128 [::]:22 [::]:*
And columns again merged. Here is how to reproduced it without subshell (just make your terminal smaller):
root@local-test-42-bionic-default:~# stty size
30 85
root@local-test-42-bionic-default:~# /bin/ss --numeric --listening --tcp
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 32 10.120.149.1:53 0.0.0.0:*
LISTEN 0 128 127.0.0.53%lo:53 0.0.0.0:*
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 128 127.0.0.1:8443 0.0.0.0:*
LISTEN 0 32 [fd42:d8d2:8099:e96e::1]:53 [::]:*
LISTEN 0 32 [fe80::d0d2:7eff:fefd:8273]%lxdbr0:53 [::]:*
LISTEN 0 128 [::]:22 [::]:*
root@local-test-42-bionic-default:~# stty size
30 84
root@local-test-42-bionic-default:~# /bin/ss --numeric --listening --tcp
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN0 32 10.120.149.1:53 0.0.0.0:*
LISTEN0 128 127.0.0.53%lo:53 0.0.0.0:*
LISTEN0 128 0.0.0.0:22 0.0.0.0:*
LISTEN0 128 127.0.0.1:8443 0.0.0.0:*
LISTEN0 32 [fd42:d8d2:8099:e96e::1]:53 [::]:*
LISTEN0 32 [fe80::d0d2:7eff:fefd:8273]%lxdbr0:53 [::]:*
LISTEN0 128 [::]:22 [::]:*
Probably the main problem are the long addresses in column "Local Addres"? After some digging I found this commit:
While rendering columns, we use a local variable to keep track of the field currently being printed, without touching current_field, which is used for buffering.
Use the right pointer to access the left delimiter for the current column, instead of always printing the left delimiter for the last buffered field, which is usually an empty string.
This fixes an issue especially visible on narrow terminals, where some columns might be displayed without separation.
Reported-by: YoyPa [email protected] Fixes: 691bd85 ("ss: Buffer raw fields first, then render them as a table") Signed-off-by: Stefano Brivio [email protected] Tested-by: YoyPa [email protected] Signed-off-by: Stephen Hemminger [email protected]
This looks suspiciously similar to our case here. I don't sure in which versions exactly problem came. But I guess, that bionic wouldn't be getting iproute >=5.1 in any time soon (or ever?).
@philpep, after all that, what are our options here? Only fallback to netstat if it's available (or use it first)? 0_o
Looks like this "merging" issue is fixed in 5.1 (builded "backport" of package just for test):
root@local-test-42-bionic-default:~/iproute2-5.1.0# dpkg -l iproute2
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===========================-==================-==================-============================================================
ii iproute2 5.1.0-1ubuntu2 amd64 networking and traffic control tools
root@local-test-42-bionic-default:~/iproute2-5.1.0# apt-cache policy iproute2
iproute2:
Installed: 5.1.0-1ubuntu2
Candidate: 5.1.0-1ubuntu2
Version table:
*** 5.1.0-1ubuntu2 100
100 /var/lib/dpkg/status
4.18.0-1ubuntu2~ubuntu18.04.1 100
100 http://ru.archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages
4.15.0-2ubuntu1 500
500 http://ru.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
root@local-test-42-bionic-default:~/iproute2-5.1.0# echo "$(/bin/ss --numeric --listening --tcp)"
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 32 10.120.149.1:53 0.0.0.0:*
LISTEN 0 128 127.0.0.53%lo:53 0.0.0.0:*
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 128 127.0.0.1:8443 0.0.0.0:*
LISTEN 0 32 [fd42:d8d2:8099:e96e::1]:53 [::]:*
LISTEN 0 32 [fe80::d0d2:7eff:fefd:8273]%lxdbr0:53 [::]:*
LISTEN 0 128 [::]:22 [::]:*
So, after that "ugly" fix our tests were successful. But nonetheless, I think, that it isn't a good idea to bring package from "devel" release.
Any other ideas?
Yep, it's definetly the commit mentioned above. Quickly builded 4.18 (from bionic-backports) and problem gone:
root@local-test-42-bionic-default:~# cat /root/iproute-width.patch
diff --git a/misc/ss.c b/misc/ss.c
index c8970438..4d12fb5d 100644
--- a/misc/ss.c
+++ b/misc/ss.c
@@ -1260,7 +1260,7 @@ static void render(void)
while (token) {
/* Print left delimiter only if we already started a line */
if (line_started++)
- printed = printf("%s", current_field->ldelim);
+ printed = printf("%s", f->ldelim);
else
printed = 0;
root@local-test-42-bionic-default:~# echo "$(/bin/ss --numeric --listening --tcp)"
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 127.0.0.53%lo:53 0.0.0.0:*
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 128 [::]:22 [::]:*
root@local-test-42-bionic-default:~# dpkg -l iproute2
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=======================-================-================-===================================================
ii iproute2 4.18.0-1ubuntu2~ amd64 networking and traffic control tools
But it's still a bit of "too hackish" way to solve "failed test" issue. Don't you think? 8)