ibm_zos_core
ibm_zos_core copied to clipboard
ansible-collections
[Enabler] Propose a solution for the UTF-8 warning
Is there an existing issue for this?
- [X] There are no existing issues.
Ansible module
No response
Enabler description
This is part of collaboration #983.
Propose a solution either for ibm_zos_core or one for ansible-core or a work around that does not interrupt playbook execution.
I discovered that the encoding of the content that Ansible deals with doesn't seem to matter so much as related to the "non-utf8 char deprecation warning" as long as the data is kept out of stdout/stderr.
I started experimenting with the zos_mvs_raw module because it can have output sent to stdout via dd_output or to a data set or something else via dd_data_set or dd_unix. Further, there are configs available to specify the resulting encoding.
With dd_output specified and EBCDIC set as the resulting encoding, the output of the module is sent to stdout in EBCDIC, captured by Ansible, and returned to the controller along with the deprecation warning about non-utf8 chars.
However, the same operation, but sending output to a data set does NOT cause the warning, even with increased levels of verbosity!! Interestingly enough, changing the verbosity changed the location of the deprecation warning. When any level beyond the default was set, the debug task no longer raised the deprecation warning, and also, the original zos_mvs_raw task raised the warning TWICE
-
playbook and output for
zos_mvs_rawwithdd_output:- name: mvs raw - uptime - output to stdout zos_mvs_raw: program_name: bpxbatch parm: "SH uptime" dds: - dd_output: dd_name: stdout return_content: type: text response_encoding: "ibm-1047" register: output - name: print debug: var: outputnon verbosity output:
TASK [mvs raw - uptime - output to stdout] **************************************************************************************************************************************************************************************************************************
[DEPRECATION WARNING]: Module "ibm.ibm_zos_core.zos_mvs_raw" returned non UTF-8 data in the JSON response. This will become an error in the future. This feature will be removed in version 2.18. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
changed: [zvm]
TASK [print] ********************************************************************************************************************************************************************************************************************************************************
[DEPRECATION WARNING]: Non UTF-8 encoded data replaced with "?" while displaying text to stdout/stderr, this is temporary and will become an error. This feature will be removed in version 2.18. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
ok: [zvm] => {
"output": {
"backups": [],
"changed": true,
"dd_names": [
{
"byte_count": 314,
"content": [
"\u0015?@?@?@m?m?@?z@?\u0015?@?@?@m?m?m?@?z@?\u0015?z@a?@?@?@?@?@㤅@Ѥ?@?@?z?z?@?@?\u0015?z@薤@?@?@?@?@م?@?a?@?K?@?\u0015?@?@?@?@?o@M恙@ǁ?k@?]\u0015?@Ⅳ@?@a?a?\u0015\u0015?z?@@?@?@?M?]k@?z?k@@?@?k@@?@?z@?K?k@?K?k@?K?\u0015"
],
"dd_name": "stdout",
"name": "KETAN.P7304619.T0046499.C0000000",
"record_count": 1
}
],
"failed": false,
"ret_code": {
"code": 0
}
}
}
-
playbook and output for
zos_mvs_rawwithdd_data_set:- name: mvs raw - uptime - output into data set. zos_mvs_raw: program_name: bpxbatch parm: "SH uptime" dds: - dd_data_set: dd_name: stdout data_set_name: "{{ DEFAULT_DATA_SET_NAME }}" type: seq disposition: new replace: yes return_content: type: text response_encoding: "ibm-1047" register: output - name: print debug: var: outputnon-verbosity output:
TASK [mvs raw - uptime - output into data set.] *********************************************************************************************************************************************************************************************************************
changed: [zvm]
TASK [print] ********************************************************************************************************************************************************************************************************************************************************
ok: [zvm] => {
"output": {
"backups": [],
"changed": true,
"dd_names": [
{
"byte_count": 315,
"content": [
"Ĩ?@?@?@m?m?@?z@?\u0015Ĩ?@?@?@m?m?m?@?z@?\u0015Ֆ?z@a?@?@?@?@?@㤅@Ѥ?@?@?z?z?@?@?\u0015Ֆ?z@薤@?@?@?@?@م?@?a?@?K?@?\u0015∁?@?@?@?@?o@M恙@ǁ?k@?]\u0015?@Ⅳ@?@a?a?\u0015?z?@@?@?@?M?]k@?z?k@@?@?k@@?@?z@?K?k@?K?k@?K?\u0015"
],
"dd_name": "stdout",
"name": "KETAN.ANSIBLE.TESTDS",
"record_count": 1
}
],
"failed": false,
"ret_code": {
"code": 0
}
}
}
This area needs more exploration, however, a possible approach may be to comb all the modules for places where stdout/stderr may be affected, and force those to UTF-8 or base64 the content as suggested by ansible-core. In my opinion, this approach will not compromise functionality (at least not much).
See following comment for verbose output (-vvv).
- verbose output for
dd_output
TASK [Set facts] ****************************************************************************************************************************************************************************************************************************************************
task path: < ..path.. >
ok: [zvm] => {
"ansible_facts": {
"DEFAULT_DATA_SET_NAME": "KETAN.ANSIBLE.TESTDS"
},
"changed": false
}
TASK [mvs raw - uptime - output to stdout] **************************************************************************************************************************************************************************************************************************
task path: < .. path .. >
Using module file< ..path ..>/.ansible/collections/ansible_collections/ibm/ibm_zos_core/plugins/modules/zos_mvs_raw.py
Pipelining is enabled.
<url> ESTABLISH SSH CONNECTION FOR USER: KETAN
<url> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="KETAN"' -o ConnectTimeout=10 -o 'ControlPath="//.ansible/cp/64a13ee7bc"' <url> '/bin/sh -c '"'"'_BPXK_AUTOCVT=ON _CEE_RUNOPTS='"'"'"'"'"'"'"'"'FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)'"'"'"'"'"'"'"'"' _TAG_REDIR_ERR=txt _TAG_REDIR_IN=txt _TAG_REDIR_OUT=txt PATH=//zoautil/1.2.5.8/bin://venvs/3.11/zoau-1.2.5.8/bin:/bin:/usr/sbin:/var/bin:. LIBPATH=/zoautil/1.2.5.8/lib://venvs/3.11/zoau-1.2.5.8/lib:/lib:/usr/lib:. LANG=C PYTHONSTDINENCODING=cp1047 //venvs/3.11/zoau-1.2.5.8/bin/python3 && sleep 0'"'"''
<url> (0, b'\n{"changed": true, "dd_names": [{"dd_name": "stdout", "name": "KETAN.P7304632.T0585585.C0000000", "content": ["\\u0015\\udca8\\udc95\\udc81\\udc94\\udc89\\udc83@\\udca2\\udc85\\udca3\\udca3\\udc89\\udc95\\udc87@\\udc96\\udc86@m\\udcc2\\udcd7\\udce7m\\udce2\\udcc8\\udcc1\\udcd9\\udcc5\\udcc1\\udce2@\\udcc9\\udce2z@\\udce8\\udcc5\\udce2\\u0015\\udca8\\udc95\\udc81\\udc94\\udc89\\udc83@\\udca2\\udc85\\udca3\\udca3\\udc89\\udc95\\udc87@\\udc96\\udc86@m\\udcc2\\udcd7\\udce7m\\udce2\\udcd7\\udcc1\\udce6\\udcd5m\\udce2\\udcc3\\udcd9\\udcc9\\udcd7\\udce3@\\udcc9\\udce2z@\\udce8\\udcc5\\udce2\\u0015\\udc96\\udca3\\udc85z@a\\udc89\\udc83\\udca3@\\udca9\\udcc6\\udce2@\\udca2\\udc85\\udca3@\\udc82\\udca4\\udc89\\udc93\\udca3@\\udc96\\udc95@\\u3905@\\u0464\\udc93@\\udcf9@\\udcf1\\udcf7z\\udcf0\\udcf5z\\udcf4\\udcf3@\\udcc5\\udcc4\\udce3@\\udcf2\\udcf0\\udcf1\\udcf3\\u0015\\udc96\\udca3\\udc85z@\\u85a4@\\udc81\\udc99\\udc85@\\udc99\\udca4\\udc95\\udc95\\udc89\\udc95\\udc87@\\udc96\\udc95@\\udca9\\udcd6\\udce2@\\u0645\\udc93\\udc85\\udc81\\udca2\\udc85@\\udca9a\\udcd6\\udce2@\\udcf0\\udcf5K\\udcf0\\udcf0@\\udcf0\\udcf2\\u0015\\udc88\\udc81\\udc93\\udc93@\\udca6\\udc85@\\udc97\\udc93\\udc81\\udca8@\\udc81@\\udc87\\udc81\\udc94\\udc85o@M\\u6059@\\u01c1\\udc94\\udc85\\udca2k@\\udcf1\\udcf9\\udcf8\\udcf3]\\u0015\\udcd5\\udcd7\\udce4\\udce3\\udcd9\\udcc3@\\u2163@\\udca3\\udc96@a\\udc85\\udca3\\udc83a\\udc89\\udc95\\udc97\\udca4\\udca3\\udc99\\udc83\\u0015\\u0015\\udcf6z\\udcf4\\udcf9\\udcd7\\udcd4@@\\udca4\\udc97@\\udcf1@\\udc84\\udc81\\udca8M\\udca2]k@\\udcf1\\udcf0z\\udcf3\\udcf5k@@\\udcf2@\\udca4\\udca2\\udc85\\udc99\\udca2k@@\\udc93\\udc96\\udc81\\udc84@\\udc81\\udca5\\udc85\\udc99\\udc81\\udc87\\udc85z@\\udcf0K\\udcf0\\udcf0k@\\udcf0K\\udcf0\\udcf0k@\\udcf0K\\udcf0\\udcf0\\u0015"], "record_count": 1, "byte_count": 315}], "ret_code": {"code": 0}, "backups": [], "invocation": {"module_args": {"program_name": "bpxbatch", "parm": "SH uptime", "dds": [{"dd_output": {"dd_name": "stdout", "return_content": {"type": "text", "response_encoding": "ibm-1047", "src_encoding": "ibm-1047"}}, "dd_data_set": null, "dd_unix": null, "dd_input": null, "dd_vio": null, "dd_concat": null, "dd_dummy": null}], "auth": false, "verbose": false, "tmp_hlq": null}}}\n', b'')
[DEPRECATION WARNING]: Module "ibm.ibm_zos_core.zos_mvs_raw" returned non UTF-8 data in the JSON response. This will become an error in the future. This feature will be removed in version 2.18. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
[DEPRECATION WARNING]: Non UTF-8 encoded data replaced with "?" while displaying text to stdout/stderr, this is temporary and will become an error. This feature will be removed in version 2.18. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
changed: [zvm] => {
"backups": [],
"changed": true,
"dd_names": [
{
"byte_count": 315,
"content": [
"\u0015?@?@?@m?m?@?z@?\u0015?@?@?@m?m?m?@?z@?\u0015?z@a?@?@?@?@?@㤅@Ѥ?@?@?z?z?@?@?\u0015?z@薤@?@?@?@?@م?@?a?@?K?@?\u0015?@?@?@?@?o@M恙@ǁ?k@?]\u0015?@Ⅳ@?@a?a?\u0015\u0015?z?@@?@?@?M?]k@?z?k@@?@?k@@?@?z@?K?k@?K?k@?K?\u0015"
],
"dd_name": "stdout",
"name": "KETAN.P7304632.T0585585.C0000000",
"record_count": 1
}
],
"invocation": {
"module_args": {
"auth": false,
"dds": [
{
"dd_concat": null,
"dd_data_set": null,
"dd_dummy": null,
"dd_input": null,
"dd_output": {
"dd_name": "stdout",
"return_content": {
"response_encoding": "ibm-1047",
"src_encoding": "ibm-1047",
"type": "text"
}
},
"dd_unix": null,
"dd_vio": null
}
],
"parm": "SH uptime",
"program_name": "bpxbatch",
"tmp_hlq": null,
"verbose": false
}
},
"ret_code": {
"code": 0
}
}
TASK [print] ********************************************************************************************************************************************************************************************************************************************************
task path: /Users/ketan/dev/playbooks/npXzoau12X.yml:61
ok: [zvm] => {
"output": {
"backups": [],
"changed": true,
"dd_names": [
{
"byte_count": 315,
"content": [
"\u0015?@?@?@m?m?@?z@?\u0015?@?@?@m?m?m?@?z@?\u0015?z@a?@?@?@?@?@㤅@Ѥ?@?@?z?z?@?@?\u0015?z@薤@?@?@?@?@م?@?a?@?K?@?\u0015?@?@?@?@?o@M恙@ǁ?k@?]\u0015?@Ⅳ@?@a?a?\u0015\u0015?z?@@?@?@?M?]k@?z?k@@?@?k@@?@?z@?K?k@?K?k@?K?\u0015"
],
"dd_name": "stdout",
"name": "KETAN.P7304632.T0585585.C0000000",
"record_count": 1
}
],
"failed": false,
"ret_code": {
"code": 0
}
}
}
- verbose output for
dd_data_set
TASK [mvs raw - uptime - output into data set.] *********************************************************************************************************************************************************************************************************************
task path: <..path..>
Using module file /<..path..>ansible/collections/ansible_collections/ibm/ibm_zos_core/plugins/modules/zos_mvs_raw.py
Pipelining is enabled.
<url> ESTABLISH SSH CONNECTION FOR USER: KETAN
<url> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="KETAN"' -o ConnectTimeout=10 -o 'ControlPath="//.ansible/cp/64a13ee7bc"' url '/bin/sh -c '"'"'_BPXK_AUTOCVT=ON _CEE_RUNOPTS='"'"'"'"'"'"'"'"'FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)'"'"'"'"'"'"'"'"' _TAG_REDIR_ERR=txt _TAG_REDIR_IN=txt _TAG_REDIR_OUT=txt PATH=//zoautil/1.2.5.8/bin:/venvs/3.11/zoau-1.2.5.8/bin:/bin:/usr/sbin:/var/bin:. LIBPATH=//zoautil/1.2.5.8/lib:/venvs/3.11/zoau-1.2.5.8/lib:/lib:/usr/lib:. LANG=C PYTHONSTDINENCODING=cp1047 venvs/3.11/zoau-1.2.5.8/bin/python3 && sleep 0'"'"''
<url> (0, b'\n{"changed": true, "dd_names": [{"dd_name": "stdout", "name": "KETAN.ANSIBLE.TESTDS", "content": ["\\u0128\\udc95\\udc81\\udc94\\udc89\\udc83@\\udca2\\udc85\\udca3\\udca3\\udc89\\udc95\\udc87@\\udc96\\udc86@m\\udcc2\\udcd7\\udce7m\\udce2\\udcc8\\udcc1\\udcd9\\udcc5\\udcc1\\udce2@\\udcc9\\udce2z@\\udcd5\\udcd6\\u0015\\u0128\\udc95\\udc81\\udc94\\udc89\\udc83@\\udca2\\udc85\\udca3\\udca3\\udc89\\udc95\\udc87@\\udc96\\udc86@m\\udcc2\\udcd7\\udce7m\\udce2\\udcd7\\udcc1\\udce6\\udcd5m\\udce2\\udcc3\\udcd9\\udcc9\\udcd7\\udce3@\\udcc9\\udce2z@\\udcd5\\udcd6\\u0015\\u0556\\udca3\\udc85z@a\\udc89\\udc83\\udca3@\\udca9\\udcc6\\udce2@\\udca2\\udc85\\udca3@\\udc82\\udca4\\udc89\\udc93\\udca3@\\udc96\\udc95@\\u3905@\\u0464\\udc93@\\udcf9@\\udcf1\\udcf7z\\udcf0\\udcf5z\\udcf4\\udcf3@\\udcc5\\udcc4\\udce3@\\udcf2\\udcf0\\udcf1\\udcf3\\u0015\\u0556\\udca3\\udc85z@\\u85a4@\\udc81\\udc99\\udc85@\\udc99\\udca4\\udc95\\udc95\\udc89\\udc95\\udc87@\\udc96\\udc95@\\udca9\\udcd6\\udce2@\\u0645\\udc93\\udc85\\udc81\\udca2\\udc85@\\udca9a\\udcd6\\udce2@\\udcf0\\udcf5K\\udcf0\\udcf0@\\udcf0\\udcf2\\u0015\\u2201\\udc93\\udc93@\\udca6\\udc85@\\udc97\\udc93\\udc81\\udca8@\\udc81@\\udc87\\udc81\\udc94\\udc85o@M\\u6059@\\u01c1\\udc94\\udc85\\udca2k@\\udcf1\\udcf9\\udcf8\\udcf3]\\u0015\\udcc9\\udcd5\\udcd7\\udce4\\udce3\\udcd9\\udcc3@\\u2163@\\udca3\\udc96@a\\udc85\\udca3\\udc83a\\udc89\\udc95\\udc97\\udca4\\udca3\\udc99\\udc83\\u0015\\udcf0\\udcf6z\\udcf4\\udcf9\\udcd7\\udcd4@@\\udca4\\udc97@\\udcf1@\\udc84\\udc81\\udca8M\\udca2]k@\\udcf1\\udcf0z\\udcf3\\udcf5k@@\\udcf2@\\udca4\\udca2\\udc85\\udc99\\udca2k@@\\udc93\\udc96\\udc81\\udc84@\\udc81\\udca5\\udc85\\udc99\\udc81\\udc87\\udc85z@\\udcf0K\\udcf0\\udcf0k@\\udcf0K\\udcf0\\udcf0k@\\udcf0K\\udcf0\\udcf0\\u0015"], "record_count": 1, "byte_count": 312}], "ret_code": {"code": 0}, "backups": [], "invocation": {"module_args": {"program_name": "bpxbatch", "parm": "SH uptime", "dds": [{"dd_data_set": {"dd_name": "stdout", "data_set_name": "KETAN.ANSIBLE.TESTDS", "type": "seq", "disposition": "new", "replace": true, "return_content": {"type": "text", "response_encoding": "ibm-1047", "src_encoding": "ibm-1047"}, "reuse": false, "backup": false, "disposition_normal": null, "disposition_abnormal": null, "space_type": null, "space_primary": null, "space_secondary": null, "volumes": null, "sms_management_class": null, "sms_storage_class": null, "sms_data_class": null, "block_size": null, "directory_blocks": null, "key_label": null, "encryption_key_1": null, "encryption_key_2": null, "key_length": null, "key_offset": null, "record_length": null, "record_format": null}, "dd_unix": null, "dd_input": null, "dd_output": null, "dd_vio": null, "dd_concat": null, "dd_dummy": null}], "auth": false, "verbose": false, "tmp_hlq": null}}}\n', b'')
changed: [zvm] => {
"backups": [],
"changed": true,
"dd_names": [
{
"byte_count": 312,
"content": [
"Ĩ?@?@?@m?m?@?z@?\u0015Ĩ?@?@?@m?m?m?@?z@?\u0015Ֆ?z@a?@?@?@?@?@㤅@Ѥ?@?@?z?z?@?@?\u0015Ֆ?z@薤@?@?@?@?@م?@?a?@?K?@?\u0015∁?@?@?@?@?o@M恙@ǁ?k@?]\u0015?@Ⅳ@?@a?a?\u0015?z?@@?@?@?M?]k@?z?k@@?@?k@@?@?z@?K?k@?K?k@?K?\u0015"
],
"dd_name": "stdout",
"name": "KETAN.ANSIBLE.TESTDS",
"record_count": 1
}
],
"invocation": {
"module_args": {
"auth": false,
"dds": [
{
"dd_concat": null,
"dd_data_set": {
"backup": false,
"block_size": null,
"data_set_name": "KETAN.ANSIBLE.TESTDS",
"dd_name": "stdout",
"directory_blocks": null,
"disposition": "new",
"disposition_abnormal": null,
"disposition_normal": null,
"encryption_key_1": null,
"encryption_key_2": null,
"key_label": null,
"key_length": null,
"key_offset": null,
"record_format": null,
"record_length": null,
"replace": true,
"return_content": {
"response_encoding": "ibm-1047",
"src_encoding": "ibm-1047",
"type": "text"
},
"reuse": false,
"sms_data_class": null,
"sms_management_class": null,
"sms_storage_class": null,
"space_primary": null,
"space_secondary": null,
"space_type": null,
"type": "seq",
"volumes": null
},
"dd_dummy": null,
"dd_input": null,
"dd_output": null,
"dd_unix": null,
"dd_vio": null
}
],
"parm": "SH uptime",
"program_name": "bpxbatch",
"tmp_hlq": null,
"verbose": false
}
},
"ret_code": {
"code": 0
}
}
TASK [print] ********************************************************************************************************************************************************************************************************************************************************
task path: <..path..>
ok: [zvm] => {
"output": {
"backups": [],
"changed": true,
"dd_names": [
{
"byte_count": 312,
"content": [
"Ĩ?@?@?@m?m?@?z@?\u0015Ĩ?@?@?@m?m?m?@?z@?\u0015Ֆ?z@a?@?@?@?@?@㤅@Ѥ?@?@?z?z?@?@?\u0015Ֆ?z@薤@?@?@?@?@م?@?a?@?K?@?\u0015∁?@?@?@?@?o@M恙@ǁ?k@?]\u0015?@Ⅳ@?@a?a?\u0015?z?@@?@?@?M?]k@?z?k@@?@?k@@?@?z@?K?k@?K?k@?K?\u0015"
],
"dd_name": "stdout",
"name": "KETAN.ANSIBLE.TESTDS",
"record_count": 1
}
],
"failed": false,
"ret_code": {
"code": 0
}
}
}
Turns out, the above is WRONG!!
The order matters!!
In the above playbook, the tasks calling dd_output were first and raised the deprecation warning while the tasks calling dd_data_set went second and did not raise the warning. HOWEVER, switching the ordering, so dd_data_set tasks first and then dd_output tasks to follow changes the behavior -- with this configuration of the playbook, the dd_data_set tasks raise the depreceation warning and the dd_output tasks do not raise the same warning which they raised in the other playbook configuration
The investigation for this item is ongoing (may break it out into a separate issue). I've been focusing on zos_mvs_raw and its output.
I took a look at the response_encoding parameter (a sub-option of return_content) to see if one aspect of the issue could be addressed by a doc-change and defining it as incorrect user behavior to put an ebcdic encoding there.
Eg:
- name: List data sets matching pattern in catalog,
return output to user, but don't store in persistent storage.
Return the contents of the file in encoding IBM-1047,
while the file is encoded in ISO8859-1.
zos_mvs_raw:
program_name: idcams
auth: true
dds:
- dd_output:
dd_name: sysprint
return_content:
type: text
response_encoding: iso8859-1 <--------- here
Defining the response_encoding as ebcdic will cause the "non-utf8 chars found in stdout/stderr" warning and eventual error. To me, this approach fits the bill as it would not interfere with operations on the z/OS side. The value of response_encoding seems to be invoked only when retrieving the dd output from the data set after the system has already put the content there.
What remains to be seen is how to handle the return content for a unix file if the content is encoded in ebcdic.
The points here have been reduced from 5 pts to 0 pts. Those 5 pts are captured in the issues listed below around exploring specific modules and use cases. As this work item expands, additional issues will be created and the list will grow (as post-plan). The following two issues should NOT be considered post-plan points.
- #1532
- #1533
Many ibm.ibm_zos_core modules call the AnsibleModule.run_command() method to run commands on the remote z/OS system (eg mvscmd).
The default “error handler” for the run_command is set to “surrogate_or_strict”, but it can be overridden via an arg.
-
default= “surrogate_or_strict”
-
"The default is surrogate_or_strict which means that the bytes will be decoded using the surrogateescape error handler if available (available on all python3 versions we support) otherwise a UnicodeError traceback will be raised. (from ansible docs)
-
"USER=IBMUSER NAME=? OWNER=IBMUSER CREATED=00.207 "+ raises WARNING!!
-
-
my current fav= “backslashreplace”
-
"USER=IBMUSER NAME=\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f\\x9f OWNER=IBMUSER CREATED=00.207 ",
-
-
other options:
-
“ignore”
-
"USER=IBMUSER NAME= OWNER=IBMUSER CREATED=00.207 ",
-
-
“replace” (with replacement char, the ‘?’ inside the box)
-
"USER=IBMUSER NAME=�������������������� OWNER=IBMUSER CREATED=00.207 ",
-
-
“xmlcharrefreplace” — Ansible doesn’t like this one..
-
"msg": "An unexpected error occurred: TypeError(\"don't know how to handle UnicodeDecodeError in error callback\")"
-
-
My recommendation is to override the errors algorithm from "surrogate_or_strict" to "backslashreplace", which should replace any non-UTF-8 output with \\ and the hex value.
Two potential issues:
- could there be any data integrity issues with this approach? -- since the raw bytes / binary potentially changes based on the defined charset, is there a way that those changes negatively affect/break any functions? Thusfar, the investigation has looked primarily at output returned to the ansible controller.
- can any of the escape values/chars themselves be non-utf-8 and cause failure?