Problem with string separator in SSHReader.readuntil()

[MazokuMaxy]

Starting with version 2.8.1 its seems SSHReader.readuntil() is not working when separator is long string.

in short output = await stdout.readuntil(separator='#') is working

output = await stdout.readuntil(separator='(config)#') is not working since 2.8.1

output = await stdout.readuntil(separator=('(config)#',)) is working

[ronf]

Based on the information above, I'm not able to reproduce this here. I get the same (correct) result for all three of these examples. What sort of error are you seeing?

Also, is this working for you with releases prior to 2.8.1? There were some type annotation changes which came in just after the 2.8.1 release (in 2.9.0), but before that the previous change related to readuntil() goes back to around 2.2.0 almost a year earlier.

[MazokuMaxy]

I did some research on my test cisco and different versions of asyncssh

Python 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> import asyncssh
>>> import async_timeout
>>> 
>>> async def script():
...     print(asyncssh.__version__)
...     async with async_timeout.timeout(10):
...         conn = await asyncssh.connect(
...             host='cisco-test',
...             port=22,
...             username='admin',
...             password='1234',
...             known_hosts=None,
...             kbdint_auth=False,
...             encryption_algs=['aes256-cbc', 'aes256-ctr',],
...             kex_algs=['diffie-hellman-group14-sha1',],
...             login_timeout=60,
...         )
...         stdin, stdout, stderr = await conn.open_session()
...         print(await stdout.readuntil(separator='#'))
...         stdin.write(data='configure terminal\r')
...         print(await stdout.readuntil(separator='#'))
... 
>>> asyncio.run(script())
2.11.0

cisco-test#
configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
cisco-test(config)#
>>> exit()

Python 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> import asyncssh
>>> import async_timeout
>>> 
>>> async def script():
...     print(asyncssh.__version__)
...     async with async_timeout.timeout(10):
...         conn = await asyncssh.connect(
...             host='cisco-test',
...             port=22,
...             username='admin',
...             password='1234',
...             known_hosts=None,
...             kbdint_auth=False,
...             encryption_algs=['aes256-cbc', 'aes256-ctr',],
...             kex_algs=['diffie-hellman-group14-sha1',],
...             login_timeout=60,
...         )
...         stdin, stdout, stderr = await conn.open_session()
...         print(await stdout.readuntil(separator='#'))
...         stdin.write(data='configure terminal\r')
...         print(await stdout.readuntil(separator='(config)#'))
... 
>>> asyncio.run(script())
2.11.0

cisco-test#
Traceback (most recent call last):
  File "<stdin>", line 18, in script
  File "/home/test/.local/lib/python3.10/site-packages/asyncssh/stream.py", line 205, in readuntil
    return await self._session.readuntil(separator, self._datatype)
  File "/home/test/.local/lib/python3.10/site-packages/asyncssh/stream.py", line 627, in readuntil
    await self._block_read(datatype)
  File "/home/test/.local/lib/python3.10/site-packages/asyncssh/stream.py", line 385, in _block_read
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "<stdin>", line 3, in script
  File "/home/test/.local/lib/python3.10/site-packages/async_timeout/__init__.py", line 129, in __aexit__
    self._do_exit(exc_type)
  File "/home/test/.local/lib/python3.10/site-packages/async_timeout/__init__.py", line 212, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError
>>> exit()

Python 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> import asyncssh
>>> import async_timeout
>>> 
>>> async def script():
...     print(asyncssh.__version__)
...     async with async_timeout.timeout(10):
...         conn = await asyncssh.connect(
...             host='cisco-test',
...             port=22,
...             username='admin',
...             password='1234',
...             known_hosts=None,
...             kbdint_auth=False,
...             encryption_algs=['aes256-cbc', 'aes256-ctr',],
...             kex_algs=['diffie-hellman-group14-sha1',],
...             login_timeout=60,
...         )
...         stdin, stdout, stderr = await conn.open_session()
...         print(await stdout.readuntil(separator='#'))
...         stdin.write(data='configure terminal\r')
...         print(await stdout.readuntil(separator=('(config)#',)))
... 
>>> asyncio.run(script())
2.11.0

cisco-test#
configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
cisco-test(config)#
>>> exit()

Python 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> import asyncssh
/home/test/.local/lib/python3.10/site-packages/asyncssh/crypto/cipher.py:29: CryptographyDeprecationWarning: Blowfish has been deprecated
  from cryptography.hazmat.primitives.ciphers.algorithms import Blowfish, CAST5
/home/test/.local/lib/python3.10/site-packages/asyncssh/crypto/cipher.py:29: CryptographyDeprecationWarning: CAST5 has been deprecated
  from cryptography.hazmat.primitives.ciphers.algorithms import Blowfish, CAST5
/home/test/.local/lib/python3.10/site-packages/asyncssh/crypto/cipher.py:30: CryptographyDeprecationWarning: SEED has been deprecated
  from cryptography.hazmat.primitives.ciphers.algorithms import SEED, TripleDES
>>> import async_timeout
>>> 
>>> async def script():
...     print(asyncssh.__version__)
...     async with async_timeout.timeout(10):
...         conn = await asyncssh.connect(
...             host='cisco-test',
...             port=22,
...             username='admin',
...             password='1234',
...             known_hosts=None,
...             kbdint_auth=False,
...             encryption_algs=['aes256-cbc', 'aes256-ctr',],
...             kex_algs=['diffie-hellman-group14-sha1',],
...             login_timeout=60,
...         )
...         stdin, stdout, stderr = await conn.open_session()
...         print(await stdout.readuntil(separator='#'))
...         stdin.write(data='configure terminal\r')
...         print(await stdout.readuntil(separator='(config)#'))
... 
>>> asyncio.run(script())
2.8.1

cisco-test#
configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
cisco-test(config)#
>>> exit()

Python 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> import asyncssh
/home/test/.local/lib/python3.10/site-packages/asyncssh/crypto/cipher.py:29: CryptographyDeprecationWarning: Blowfish has been deprecated
  from cryptography.hazmat.primitives.ciphers.algorithms import Blowfish, CAST5
/home/test/.local/lib/python3.10/site-packages/asyncssh/crypto/cipher.py:29: CryptographyDeprecationWarning: CAST5 has been deprecated
  from cryptography.hazmat.primitives.ciphers.algorithms import Blowfish, CAST5
/home/test/.local/lib/python3.10/site-packages/asyncssh/crypto/cipher.py:30: CryptographyDeprecationWarning: SEED has been deprecated
  from cryptography.hazmat.primitives.ciphers.algorithms import SEED, TripleDES
>>> import async_timeout
>>> 
>>> async def script():
...     print(asyncssh.__version__)
...     async with async_timeout.timeout(10):
...         conn = await asyncssh.connect(
...             host='cisco-test',
...             port=22,
...             username='admin',
...             password='1234',
...             known_hosts=None,
...             kbdint_auth=False,
...             encryption_algs=['aes256-cbc', 'aes256-ctr',],
...             kex_algs=['diffie-hellman-group14-sha1',],
...             login_timeout=60,
...         )
...         stdin, stdout, stderr = await conn.open_session()
...         print(await stdout.readuntil(separator='#'))
...         stdin.write(data='configure terminal\r')
...         print(await stdout.readuntil(separator='(config)#'))
... 
>>> asyncio.run(script())
2.9.0

cisco-test#
Traceback (most recent call last):
  File "<stdin>", line 18, in script
  File "/home/test/.local/lib/python3.10/site-packages/asyncssh/stream.py", line 205, in readuntil
    return await self._session.readuntil(separator, self._datatype)
  File "/home/test/.local/lib/python3.10/site-packages/asyncssh/stream.py", line 627, in readuntil
    await self._block_read(datatype)
  File "/home/test/.local/lib/python3.10/site-packages/asyncssh/stream.py", line 385, in _block_read
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "<stdin>", line 3, in script
  File "/home/test/.local/lib/python3.10/site-packages/async_timeout/__init__.py", line 129, in __aexit__
    self._do_exit(exc_type)
  File "/home/test/.local/lib/python3.10/site-packages/async_timeout/__init__.py", line 212, in _do_exit
    raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError
>>> exit()

Without async_timeout readuntil just run forever.

[ronf]

Thanks! I did a bit more digging tonight and I think I found the issue. I think it only occurs when you have a string separator which contains characters that have special meaning in regular expressions, such as the parentheses you are including here.

I believe the following patch should fix it:

diff --git a/asyncssh/stream.py b/asyncssh/stream.py
index a8e95a4..51085f8 100644
--- a/asyncssh/stream.py
+++ b/asyncssh/stream.py
@@ -569,7 +569,7 @@ class SSHStreamSession(Generic[AnyStr]):
             separators = cast(AnyStr, '\n' if self._encoding else b'\n')
         elif isinstance(separator, (bytes, str)):
             seplen = len(separator)
-            separators = cast(AnyStr, separator)
+            separators = re.escape(cast(AnyStr, separator))
         else:
             bar = cast(AnyStr, '|' if self._encoding else b'|')
             seplist = list(cast(Iterable[AnyStr], separator))

This escaping was already in place for the code path where you provide a list of patterns, which would explain why it worked in that case in your tests.

[ronf]

This fix is now available as commit a9ed02e in the "develop" branch. Please let me know if this doesn't work for you. Thanks again for the report!

[ronf]

This change is now available in AsyncSSH 2.12.0.