Chinese name of test case can not display correct on console log

[mark26195491]

RIDE v2.0b2.dev1 running on Python 3.8.5.

Package Version

---

bcrypt 3.2.0 cffi 1.14.4 cryptography 3.3.1 numpy 1.19.5 paramiko 2.7.2 Pillow 8.1.0 pip 21.0 psutil 5.8.0 pycparser 2.20 Pygments 2.7.4 PyMySQL 1.0.2 PyNaCl 1.4.0 Pypubsub 4.0.3 pywin32 300 robotframework 3.2.2 robotframework-ride 2.0b2.dev1 robotframework-sshlibrary 3.6.0 scp 0.13.3 setuptools 47.1.0 six 1.15.0 wxPython 4.0.7.post2

[mark26195491]

Exception:

[HelioGuilherme66]

Please try with current development version. Then, try to upgrade wxPython.

[mark26195491]

Please try with current development version. Then, try to upgrade wxPython.

Currently， i have already use the dev version RIDE v2.0b2.dev1

[HelioGuilherme66]

Then, try to upgrade wxPython.

Please paste output of command: chcp

[clclclclclclcl]

[yuwy9988]

python3.9 543行不是这个

---

python3.9 543 OK is not this

[HelioGuilherme66]

@yuwy9988 Please communicate in English.

I am still waiting for the output of : chcp

Also the content of Tools>View RIDE Log.

[yuwy9988]

[HelioGuilherme66]

Need the output of command chcp in a command window. This is mine:

C:\>chcp
Active code page: 850

And you should try running robot in a command window to confirm the console log shows OK.

This is only a visual problem on RIDE, for the Console Log, not for the Output Log.

[yuwy9988]

change post request ok

[yuwy9988]

How to set when the label page disappears run tag

[HelioGuilherme66]

@mark26195491 Please retest on new version 2.0b2.

[zhujunling-nj]

RIDE v2.0b2 wxPython 4.1.1 Python 3.8.10 Windows 10

I got the error on python console when the name of test case contains Chinese: _Traceback (most recent call last): File "C:\Python38\lib\site-packages\robotide\contrib\testrunner\testrunnerplugin.py", line 575, in OnTimer self._append_to_console_log(out_buffer, source="stdout") File "C:\Python38\lib\site-packages\robotide\contrib\testrunner\testrunnerplugin.py", line 605, in _append_to_console_log self._append_text(self._console_log_ctrl, text, source) File "C:\Python38\lib\site-packages\robotide\contrib\testrunner\testrunnerplugin.py", line 626, in append_text text_ctrl.AppendText(text) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb4 in position 12: invalid start byte

I modify the site-packages\robotide\contrib\testrunner\testrunnerplugin.py, and then fix the error.

[HelioGuilherme66]

Please see my comment on #2494

[HelioGuilherme66]

This is another test case with similar problems, even applying the #2494 will not show the expected output.

*** Test Cases ***
TC1
    [Tags]    console
    Log to console    \nHello world!
    Log to console    Diacritics: éëè
    Log to console    Hello world again!
TC2
    [Tags]    console
    Log to console    \nHello world!
    Log to console    Hello world again!

[zhujunling-nj]

This is another test case with similar problems, even applying the #2494 will not show the expected output.

*** Test Cases ***
TC1
    [Tags]    console
    Log to console    \nHello world!
    Log to console    Diacritics: éëè
    Log to console    Hello world again!
TC2
    [Tags]    console
    Log to console    \nHello world!
    Log to console    Hello world again!

I set windows non-unicode to Dutch and modified encoding ['CONSOLE'] in testrunnerplugin.py to encoding ['SYSTEM'], the test case TC1 seems to be working properly.

[zhujunling-nj]

In the Dutch language Windows, OEM Code Page is 850, ANSI Code Page is 1252. The OEMCP apply to the console system calls, and the ANSICP apply to the file system calls. Read and write the robot's STDOUT seem to be file operation.

[Harm10]

@zhujunling-nj Your suggestion for encoding SYSTEM also works with me (in essence this is using locale.getpreferredencoding()) but the Chinese test case does not work properly. The Chinese characters all end up as ? characters. Is this also the case when you run it with that SYSTEM encoding?

[zhujunling-nj]

@zhujunling-nj Your suggestion for encoding SYSTEM also works with me (in essence this is using locale.getpreferredencoding()) but the Chinese test case does not work properly. The Chinese characters all end up as ? characters. Is this also the case when you run it with that SYSTEM encoding?

When I set Winodws to Chinese (code page 936), the Chinese test case runs normal, but TC1 runs abnormally. In winodws systems, you cannot support two or more non-unicode charsets simultaneously.

[zhujunling-nj]

In python 3, I use another solution (without the 2494), it seems to be working properly. Set the windows environment PYTHONIOENCODING to UTF-8, and no source code need to be modified. And this solution supports multi-languages. If this solution also applies to someone else, I will close the 2494

[Harm10]

What is the best way to set PYTHONIOENCODING to UTF-8? Should it only be set for RIDE execution or RF in general? The latter should also have effect on command line execution?

[zhujunling-nj]

I have tried two solutions:

1. Set the RIDE shortcut target to C:\Python38\pythonw.exe -c "import os; from robotide import main; os.environ['PYTHONIOENCODING'] = 'UTF-8'; main()". This is only effective for the RIDE and RF runned by RIDE.
2. Set the system environment PYTHONIOENCODING to UTF-8. This effective for all python programs.

The latter may cause an abnormal display of the python programs running on some platforms (such as Jenkins, etc.). These platforms parse the output of the program with a ansi encoding.

[zhujunling-nj]

I have tried two solutions:

1. Set the RIDE shortcut target to C:\Python38\pythonw.exe -c "import os; from robotide import main; os.environ['PYTHONIOENCODING'] = 'UTF-8'; main()". This is only effective for the RIDE and RF runned by RIDE.
2. Set the system environment PYTHONIOENCODING to UTF-8. This effective for all python programs.

The latter may cause an abnormal display of the python programs running on some platforms (such as Jenkins, etc.). These platforms parse the output of the program with a ansi encoding.

Another solution: Modity robotide\contrib\testrunner\Process.py，add one line as follows. This is only effective for the RF runned by RIDE on windows.

[Harm10]

@HelioGuilherme66 I quite like the solution in Process.py. You are then totally independent of what is set outside RIDE. Do you want me to test this?

@zhujunling-nj Maybe you have seen my own PR for testrunnerplugin.py? As this looks like a Windows problem only I tried to make the fix more specific. Maybe you could also have a look?

[HelioGuilherme66]

I will test @zhujunling-nj PR, because is already updated to be Windows only, and seems a good solution. Only today I could reproduce the problem. This has been a problem since around 2009. We fix in RIDE, but then some changes in RF brings it back.

[HelioGuilherme66]

Closed by #2494

[zhujunling-nj]

@zhujunling-nj Maybe you have seen my own PR for testrunnerplugin.py? As this looks like a Windows problem only I tried to make the fix more specific. Maybe you could also have a look?

Yes, I looked at the PR #2502. The solution for this PR and my first PR is to guess the encoding of RF's STDOUT/STDERR in pipeline mode. And the solution for my second PR is to change the encoding of RF's STDOUT/STDERR in pipeline mode. All the PR depends on the implementation of the RF, the source as follows (robot\utils\encoding.py):

def _get_console_encoding(stream):
    encoding = getattr(stream, 'encoding', None)
    if isatty(stream):
        return encoding or CONSOLE_ENCODING
    if PYTHONIOENCODING:
        return PYTHONIOENCODING
    return encoding or SYSTEM_ENCODING

If the RF can set output encoding through the command line, RIDE will not depend on the implementation of the RF.