asyncssh
asyncssh copied to clipboard
ronf
how to use proxycommand in asyncssh
I use the following command: ssh -oStrictHostKeyChecking=no -oProxyCommand='ssh -oStrictHostKeyChecking=no -T [email protected]' u180599@ and my own key id_rsa which is in ~/.ssh to connect to intel cloud and success.
Now I want to do that in asyncssh. I try
import asyncssh
async def main():
async with asyncssh.connect('ssh.devcloud.intel.com', username='guest', known_hosts=None) as conn:
async with asyncssh.connect('', username='u180599', known_hosts=None, tunnel=conn) as connection:
result = await connection.run('ls')
print(result.stdout, end='')
asyncio.run(main())
But get:
Traceback (most recent call last):
File "/home/chaowenguo/Downloads/click.py", line 51, in <module>
asyncio.run(main())
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
return future.result()
File "/home/chaowenguo/Downloads/click.py", line 8, in main
async with asyncssh.connect('', username='u180599', known_hosts=None, tunnel=conn) as connection:
File "/usr/lib/python3/dist-packages/asyncssh/misc.py", line 220, in __aenter__
self._result = await self._coro
File "/usr/lib/python3/dist-packages/asyncssh/connection.py", line 6460, in connect
return await _connect(options.host, options.port, loop, options.tunnel,
File "/usr/lib/python3/dist-packages/asyncssh/connection.py", line 214, in _connect
_, conn = await tunnel.create_connection(conn_factory, host, port)
File "/usr/lib/python3/dist-packages/asyncssh/connection.py", line 3522, in create_connection
session = await chan.connect(session_factory, remote_host, remote_port,
File "/usr/lib/python3/dist-packages/asyncssh/channel.py", line 1870, in connect
return (await self._open_tcp(session_factory, b'direct-tcpip',
File "/usr/lib/python3/dist-packages/asyncssh/channel.py", line 1862, in _open_tcp
return (await super()._open_forward(session_factory, chantype,
File "/usr/lib/python3/dist-packages/asyncssh/channel.py", line 1838, in _open_forward
packet = await super()._open(chantype, *args)
File "/usr/lib/python3/dist-packages/asyncssh/channel.py", line 633, in _open
return await self._open_waiter
asyncssh.misc.ChannelOpenError: open failed
Any idea how to do that in asyncssh?
It looks like the first call to asyncssh.connect() logging in as 'guest' is succeeding here, but there's a problem opening an SSH "direct TCP/IP" channel to use for tunneling the second call to asyncssh.connect(). My guess is that devcloud.intel.ssh.com doesn't support that and instead is looking for you to be running the tunneled SSH connection over stdin/stdout on an interactive SSH session. AsyncSSH's "tunnel" feature isn't designed to do that.
Since AsyncSSH does support ProxyCommand, you could use that to accomplish this, with either the same call out to OpenSSH to set up the outer tunnel or by running a second Python instance and invoking AsyncSSH in that instance. However, that's clearly not as efficient as running everything in a single process and single event loop.
I don't think it's possible to do this with AsyncSSH tunneling as it exists today, but it might be possible to create a subclass of SSHClientSession which could do it, without touching AsyncSSH internals. Unfortunately, to test that, I would need to be able to replicate the server environment, as it requires that the remote system basically run an 'sshd' process on stdin/stdout of the initial inbound SSH connection to ssh.devcloud.intel.com. Perhaps I can simulate this by running 'nc' as a remote command, though, at least to see if the basic concept works.
I'll get back to you after I experiment with this a bit more.
Thank you, I do not have to use asyncssh tunnel. I just want to know how to finish my job in the framework in asyncssh.
You said that:
Since AsyncSSH does support ProxyCommand, you could use that to accomplish this, with either the same call out to OpenSSH to set up the outer tunnel or by running a second Python instance and invoking AsyncSSH in that instance
Could you give me a minimal example how to use ProxyCommand in asyncssh.connect or any other function in asynssh?
It should look something like:
import asyncio, asyncssh
async def main():
async with asyncssh.connect('', username='u180599', known_hosts=None,
proxy_command='ssh -oStrictHostKeyChecking=no -T [email protected]') as conn:
result = await conn.run('ls')
print(result.stdout, end='')
asyncio.run(main())
You could also use the ProxyCommand directive in the SSH config file, and it would apply to both OpenSSH and AsyncSSH.
I try to run the example as you shown:
import asyncio, asyncssh
async def main():
async with asyncssh.connect('', username='u180599', known_hosts=None, proxy_command='ssh -oStrictHostKeyChecking=no -T [email protected]') as conn:
result = await conn.run('ls')
print(result.stdout, end='')
asyncio.run(main())
I get
TypeError: cannot unpack non-iterable NoneType object
I use asyncssh 2.14.0
full debug ouput:
Traceback (most recent call last):
File "/home/chaowenguo/async.py", line 8, in <module>
asyncio.run(main())
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
return future.result()
File "/home/chaowenguo/async.py", line 4, in main
async with asyncssh.connect('', username='u180599', known_hosts=None, proxy_command='ssh -oStrictHostKeyChecking=no -T [email protected]') as conn:
File "/home/chaowenguo/.local/lib/python3.10/site-packages/asyncssh/misc.py", line 274, in __aenter__
self._coro_result = await self._coro
File "/home/chaowenguo/.local/lib/python3.10/site-packages/asyncssh/connection.py", line 8269, in connect
return await asyncio.wait_for(
File "/usr/lib/python3.10/asyncio/tasks.py", line 408, in wait_for
return await fut
File "/home/chaowenguo/.local/lib/python3.10/site-packages/asyncssh/connection.py", line 436, in _connect
await options.waiter
File "/home/chaowenguo/.local/lib/python3.10/site-packages/asyncssh/connection.py", line 1281, in connection_made
self._connection_made()
File "/home/chaowenguo/.local/lib/python3.10/site-packages/asyncssh/connection.py", line 3271, in _connection_made
self._host, self._port = cast(HostPort, remote_peer)
TypeError: cannot unpack non-iterable NoneType object
I apply the intel cloud in https://devcloud.intel.com/oneapi/home, maybe you can apply an account on that to test the ssh connection environment.
Support for proxy_command was only added in AsyncSSH 2.7.0 (released in June of 2021), so that would explain your original issue.
The new error appears to be related to "remote_peername" not being filled in, which is expected when using proxy_command. It's triggering a problem here though because your hostname is an empty string. Try calling connect() with a hostname set and see if that helps.
thank you the proxy_command finally work. now i want to know whether there are other options in asyncssh can do the same job without explicity call ssh command like ssh -oStrictHostKeyChecking=no -T [email protected] in proxy_command
I haven't had a chance to look into getting an Intel dev cloud account yet, but here are some thoughts on this:
As it currently stands, AsyncSSH only supports "native" tunneling via direct TCP/IP sessions, and not interactive sessions over stdin/stdout. However, you might be able to do SSH over stdin/stdout with something like:
import asyncio, asyncssh, sys
tunnel_host = 'ssh.devcloud.intel.com'
tunnel_user = 'guest'
remote_host = ''
remote_user = 'u180599'
class SSHTunnel:
async def create_connection(self, protocol_factory,
_remote_host, _remote_port):
"""Open an SSH connection over an interactive SSH channel"""
conn = await asyncssh.connect(tunnel_host, username=tunnel_user)
return await conn.create_session(protocol_factory, encoding=None)
async def main():
async with asyncssh.connect(remote_host, username=remote_user,
tunnel=SSHTunnel()) as conn:
result = await conn.run('ls')
print(result.stdout, end='')
asyncio.run(main())
There might still be an issue here where it doesn't work with an empty string for a hostname. I can fix that, but in the meantime just try filling in a non-empty value.
Also, I think there might be issues cleaning up these sessions properly when they close, as there are some methods called on SSHSession objects which are not implemented yet on SSHConnection, since AsyncSSH never expected an SSHClientChannel to be the "transport" used for an SSHConnection. It is currently expecting something like SSHTCPChannel, which doesn't send these extra callbacks.
I finally got a chance to sign up for a DevCloud account, and I was able to get AsyncSSH to work without using a ProxyCommand. It required only a small change from the code I posted above, to include client keys in both SSH connect calls, and set the remote host to 'devcloud':
import asyncio, asyncssh, sys
tunnel_host = 'ssh.devcloud.intel.com'
tunnel_user = 'guest'
remote_host = 'devcloud'
remote_user = 'u180599'
client_keys = ['~/.ssh/devcloud-access-key-180599.txt']
class SSHTunnel:
async def create_connection(self, protocol_factory,
_remote_host, _remote_port):
"""Open an SSH connection over an interactive SSH channel"""
conn = await asyncssh.connect(tunnel_host, username=tunnel_user,
client_keys=client_keys)
return await conn.create_session(protocol_factory, encoding=None)
async def main():
async with asyncssh.connect(remote_host, username=remote_user,
client_keys=client_keys,
tunnel=SSHTunnel()) as conn:
result = await conn.run('ls')
print(result.stdout, end='')
asyncio.run(main())
This still may have issues when cleaning up the connection, though, as SSHClientConnection doesn't have callbacks defined for things like exit_status_received(). Handling those properly will probably require some changes in AsyncSSH, or perhaps building a wrapper object that can consume those callbacks.
Now I want to use asyncssh forward_socks with tls_client
import asyncio, asyncssh, tls_client
class SSHTunnel:
async def create_connection(self, protocol_factory, _remote_host, _remote_port):
conn = await asyncssh.connect('ssh.devcloud.intel.com', username='guest')
return await conn.create_session(protocol_factory, encoding=None)
async def main():
async with asyncssh.connect('devcloud', username='u214193', tunnel=SSHTunnel()) as conn:
await conn.forward_socks('localhost', 1080)
client = tls_client.Session()
client.proxies = 'socks5://localhost:1080'
print(client.get('https://httpbin.org/ip').json())
asyncio.run(main())
the program is hang in thie file .local/lib/python3.10/site-packages/tls_client/sessions.py
433 response = request(dumps(request_payload).encode('utf-8'))
I try
ssh -fNT -D 1080 -oStrictHostKeyChecking=no -oServerAliveInterval=60 -oProxyCommand='ssh -oStrictHostKeyChecking=no -oServerAliveInterval=60 -T [email protected]' u214193@
with
async def main():
client = tls_client.Session()
client.proxies = 'socks5://localhost:1080'
print(client.get('https://httpbin.org/ip').json())
asyncio.run(main())
working
I also try
import asyncio, asyncssh, tls_client
class SSHTunnel:
async def create_connection(self, protocol_factory, _remote_host, _remote_port):
conn = await asyncssh.connect('ssh.devcloud.intel.com', username='guest')
return await conn.create_session(protocol_factory, encoding=None)
async def main():
async with asyncssh.connect('devcloud', username='u214193', tunnel=SSHTunnel()) as conn:
await conn.forward_socks('localhost', 1080)
await asyncio.sleep(60 * 10)
asyncio.run(main())
and
curl -x socks5://localhost:1080 https://httpbin.org/ip
working
But I have no idea why asyncssh forward_socks with tls_client not working
I think your problem here is that tls_client is not an async library. As a result, you are sitting blocked in that and are no longer servicing any of the async tasks in the asyncio event queue. So, you never get a response to the SOCKS request. That's why it works if you run it outside of the event loop, in a separate process.
You can either use an asyncio-friendly version of an HTTP client (like aiohttp), or you could do something like create a separate thread for running the tls_client call. One way to do that is to use an asyncio executor. For instance, add the following function:
def tls_connect():
client = tls_client.Session()
client.proxies = 'socks5://localhost:1080'
return client.get('https://httpbin.org/ip').json()
and then change the tls_client stuff in main() to:
loop = asyncio.get_event_loop()
print(await loop.run_in_executor(None, tls_connect))
This would go right after the forward_socks() call.
Closing this due to inactivity - feel free to open a new issue if you have additional questions.