andfoy
Garbled output on Telnet Client
I'm trying to write a wrapper around Windows console apps that will allow me to connect to their stdio from network/remote clients. It is a VERY pared down concept of what Terminado does.
Each new connection gets a "reader" task that writes directly to the subprocess stdin and there is a "writer" class that monitors a global queue of pty output and forwards the bytes on to all connected clients. The connected clients are a list of writers in a global variable as is the PTY_PROC object.
There is also the pty_reader, that takes text from the PTY and puts it on the global byte queue.
I am testing using Telnet. I get some good data to the client, but some of it is garbled up. Mainly:
- when I type in characters, they are doubled up ("ddiirr")
- The prompt doesn't always line up properly with the edge of the screen
- Some times escape codes are being printed.
- It's possible to navigate all over the screen which the use of winpty is meant to restrict
So far, I've tried matching columns (80) and rows (24) and have been looking in to different terminal settings (i.e VT100, etc), but to no avail.
I'm hoping it's something simple and "obvious".
The core essentials (I believe) are here:
async def client_reader(reader, addr = ''):
# Monitor incoming data and put the bytes in to the subproc stdin
cmd = ''
while True:
try:
s = await reader.read(1024)
except asyncio.CancelledError as e:
break
PTY_PROC.write(s.decode())
print(f"client_reader ({addr}): Cancelled and tidying up...")
return(f"client_reader ({addr})")
async def write_clients():
# Watches the global CLIENTS_QUEUE and writes bytes out to
# all the clients in "WRITERS"
data = None
while True:
try:
data = await CLIENTS_QUEUE.get()
if not data:
break
for writer in WRITERS:
if data != None:
writer.write(data.encode())
await writer.drain()
except asyncio.CancelledError:
break
for writer in WRITERS:
# flush existing data to clients
if data != None:
writer.write(data.encode())
writer.write(b"\r\n\r\n### Server shutting down or restarting...")
await writer.drain()
writer.close()
await writer.wait_closed()
print(f"write_clients exiting.")
return("write_clients")
async def pty_output():
# Read chars/strings from pty and place on byte queue.
global CACHED_OUTPUT
timeout = 0.1
s = ''
while True:
try:
r, _, _ = select.select([PTY_PROC.fd], [], [], timeout)
if not r:
await asyncio.sleep(0.1)
continue
s = PTY_PROC.read(1024)
await CLIENTS_QUEUE.put(s.encode())
except asyncio.CancelledError:
print(f"pty_output: Cancelled and tidying up...")
break
if len(s) > 0: # flush any remaining data to the queue
await CLIENTS_QUEUE.put(s.encode())
return("pty_output")
Hi @Robatronic, thanks for reaching out! I have two questions regarding your use-case of pywinpty:
- Have you tried to use the library in a synchronous fashion? Without using async primitives?
- Do you know which backend is being used? Right now
ConPTYis using VT100 by default: https://github.com/andfoy/winpty-rs/blob/3d5a3a8604be35ad0ce8e37b23dd5e7ab914697e/src/pty/conpty/pty_impl.rs#L126, in which case is better to use theWinPTYbackend
- I'm working on building one presently, I'll let you know how that goes. :)
- I have tried both ConPTY and WinPTY with the same results.
Here's what I wrote:
import logging
logging.basicConfig(level=logging.DEBUG)
from winpty import PtyProcess as PtyProcessUnicode, Backend
import time
import codecs
import socketserver
pty = None
OUTPUT_CACHE = b''
class ptyTCPHandler(socketserver.BaseRequestHandler):
"""
The request handler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def setup(self):
super().setup()
# Add to client list
print(f"Adding {self.client_address} to client list")
clients.append(self)
return(0)
def handle(self):
global OUTPUT_CACHE
# self.request is the TCP socket connected to the client
# self.request.sendall("Hello there!!".encode())
self.request.sendall(OUTPUT_CACHE)
while True:
self.data = self.request.recv(1024)
logging.info(f"TCP handler recvd: {self.data}")
pty.write(self.data.decode())
try:
ch = pty.read(65536)
OUTPUT_CACHE += ch.encode()
for client in clients:
print(f"Writing {ch.encode()} to {client.client_address}")
client.request.sendall(ch.encode())
except Exception as e:
print(f"#### EXCEPTION: ptyTCPHandler: {e}")
break
def finish(self):
print(f"Removing {self.client_address} from client list..")
clients.remove(self)
if __name__ == "__main__":
subproc_name = "cmd.exe"
# subproc = "python.exe"
pty = PtyProcessUnicode.spawn(subproc_name, backend=Backend.WinPTY)
# pty = PtyProcessUnicode.spawn(subproc_name, backend=Backend.ConPTY)
pty.decoder = codecs.getincrementaldecoder('utf-8')(errors='replace')
pty.name = subproc_name
print(f"pty size: (rows, columns): {pty.getwinsize()}")
# Get first few lines from process startup
time.sleep(0.5)
OUTPUT_CACHE += pty.read(65536).encode()
# Start socket server
HOST, PORT = "localhost", 11000
# Create the server, binding to localhost on port 9999
with socketserver.TCPServer((HOST, PORT), ptyTCPHandler) as server:
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
print(f"TCP Server running on {HOST}:{PORT}")
server.serve_forever()
exit(0)
This is the output on connecting:
0;C:\Windows\System32\cmd.exeMicrosoft Windows [Version 10.0.18363.1556][9C
(c) 2019 Microsoft Corporation. All rights reserved.
[52C
D:\sandbox\epics\pyProcServ\area_51>[16C
...then, after hitting enter and typing in dir....
D:\sandbox\epics\pyProcServ\area_51>ddirir
Volume in drive D is DATA[35C
Volume Serial Number is DCBD-282A[27C
[61C
Directory of D:\sandbox\epics\pyProcServ\area_51[12C
[61C
2022-08-17 04:51 PM <DIR> .[21C
2022-08-17 04:51 PM <DIR> ..[20C
2021-12-09 06:06 PM 943 asyncio_echo_client.py
2021-12-09 06:05 PM 785 asyncio_echo_server.py
2021-12-10 12:41 AM 4,202 asyncio_subprocess.py
2021-12-18 05:26 AM 1,389 asyncio_testing.py
2022-08-10 03:58 PM 1,001 book_samples.py
2022-01-13 05:50 PM 533 common_demo_stuff.py
2022-08-12 10:58 AM 3,380 constants.py[10C
2022-01-11 01:32 AM 2,260 create_task.py
2021-12-09 10:24 PM 18 dir.bat[15C
2021-12-24 04:35 PM 854 echoProc.py[11C
2021-12-09 05:33 PM 673 echo_server.py
0;C:\Windows\System32\cmd.exe - dir861 junk.txt[14C
2022-08-10 03:53 PM 158 listin
g_13_11.py[25C
2022-08-10 04:16 PM 269 listing_13_13.py[25C
2022-08-10 04:28 PM 1,994 listing_13_14.py[25C
2022-08-09 09:04 PM 1,275 localShell.py[28C
2022-01-11 02:37 AM 892 misc.py[34C
2021-12-09 03:52 PM 2,515 pipe_client.py[27C
2021-12-09 03:51 PM 2,431 pipe_server.py[27C
2022-01-16 04:45 PM 1,200 prime_numbers.py[25C
2022-08-09 09:25 PM 5,591 ptyprocess.py[28C
2022-08-17 04:51 PM 16,769 pty_testing.py[27C
2021-12-20 04:29 PM 6,675 pynCA.py[33C
2021-12-18 10:14 AM 0 pynCA_helpers.py[25C
2022-03-24 03:52 PM 23 run_pty.bat[30C
2022-08-17 03:34 PM 38,758 server.log[31C
2022-08-17 02:57 PM 25,438 simple_server.py[25C
2021-12-09 03:49 PM 3,686 socket_stream_redirect.py[16C
2021-12-21 01:59 PM 3,963 stream_testing.py[24C
2022-08-10 04:41 PM 8,423 sys_calls.py[29C
2022-01-13 06:00 PM <DIR> templates[32C
2021-12-09 03:52 PM 635 test-socket_stream_redirect.py[11C
2022-01-27 08:09 PM 11,979 test_subprocess.py[23C
2022-01-13 08:52 PM 2,498 web_term.py[30C
2022-08-10 05:12 PM 4,538 winpsuedoterm.py[25C
2022-08-15 10:07 AM <DIR> __pycache__[30C
34 File(s) 166,609 bytes[35C
4 Dir(s) 1,703,480,389,632 bytes free[27C
0;C:\Windows\System32\cmd.exe
D:\sandbox\epics\pyProcServ\area_51>[44C
Telnet settings:
Escape Character is 'CTRL+]' Will auth(NTLM Authentication) Local echo off New line mode - Causes return key to send CR & LF Current mode: Console Will term type Preferred term type is ANSI
Looking at it, I'm starting to wonder if some of the escape codes aren't being sent as an atomic unit.....
The terminado project is very slick.
I'm trying to do similar with regular sockets rather than websockets.....
It does
I first found Terminado in Jupyterlab and dug in to find it uses pyWinPty.
I have another experiment I want to do to try and track down where the issue is.