script-server
script-server copied to clipboard
bugy
UnicodeDecodeError: 'utf-8' codec can't decode byte xxx
Hi there :) again a utf-8 against iso-8859-15 issue. Scriptserver has bash env utf-8 connects to host test.de with user oracle which has bash env de_DE.iso885915 set. Script outputs an umlaut and Scriptserver shows error:
Unexpected error occurred. Contact the administrator.
>> KILLED
The server.log says:
2021-08-26 16:19:50,663 [script_server.execution_service.INFO] Calling script #1497: ssh [email protected] /home/oracle/bin/prod2test_CS2.sh
2021-08-26 16:19:50,677 [tornado.access.INFO] 200 POST /executions/start (127.0.0.1) 31.51ms
2021-08-26 16:19:50,683 [tornado.access.INFO] 101 GET /executions/io/1497 (127.0.0.1) 0.67ms
2021-08-26 16:19:50,870 [script_server.process_popen.ERROR] Failed to read script output
Traceback (most recent call last):
File "src/execution/process_popen.py", line 71, in pipe_process_output
data = self.process.stdout.read(1)
File "/usr/lib/python3.7/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 6: invalid continuation byte
2021-08-26 16:19:50,883 [web_server.INFO] lutz disconnected
0xe4 is a german "ä" which is an output from a program that is being executed.
The only fix is to use a pipe and converting to utf-8 like so:
echo "This is a german umlaut ä" | iconv -f latin9 -t utf8
Hi @hamsterbacke, could you try using terminal mode? requires_terminal : true (but it works only on Linux) The fix, which I did in #376 works only for this mode (which is default on Linux)
Will try it. It would be nicer if the scriptserver wouldn't kill the whole process just because of an unknown character. As I filed bug #376 , it just got truncated but the script continued. Now the script gets killed. I wouldn't mind to see a strange character or the character left out but continuing the script :)
You are absolutely correct. I ticked the "Enable pseudo-terminal" box and now it accepts the non-utf8 latin9 aka iso-8859-15 characters. Sorry that I missed it the first time when you fixed it in #376 . Many thanks!