ibm_zos_core
ibm_zos_core copied to clipboard
ansible-collections
[Bug] [zos_operator] Evaluate the ability to escape command prefixes
Is there an existing issue for this?
- [X] There are no existing issues.
Bug description
Had this ansible to issue cmd
- name: "{{nodeType }} STC running check using cmd"
delegate_to: zceeZosLpar
ibm.ibm_zos_core.zos_operator:
cmd: "$D JQ''{{ lookup('vars', inventory_hostname).nodeName|upper }}''"
register: bc2Result
Note the double single quotes around the {{ and }}
A year ago this ran all ok, using older version of zOS Ansible, as shown by this output from then
TASK [ocpzcx-stc-status : bootstrap STC running check cmd result] ******************************************************************************
Friday 10 February 2023 20:46:24 +0000 (0:00:03.525) 0:01:31.173 *******
[0;32mok: [ocpboot] => { [0m
[0;32m "msg": [ [0m
[0;32m "ZZZZ 2023041 15:46:23.33 ISF031I CONSOLE ZCXPRV1 ACTIVATED", [0m
[0;32m "ZZZZ 2023041 15:46:23.33 -$D JQ'OCPDBOOT'", [0m
[0;32m "ZZZZ 2023041 15:46:23.33 STC08176 $HASP890 JOB(OCPDBOOT)", [0m
[0;32m " $HASP890 JOB(OCPDBOOT) STATUS=(AWAITING HARDCOPY),CLASS=STC,", [0m
[0;32m " $HASP890 PRIORITY=1,SYSAFF=(ANY),HOLD=(NONE)", [0m
[0;32m "ZZZZ 2023041 15:46:23.33 STC08819 $HASP890 JOB(OCPDBOOT)", [0m
[0;32m " $HASP890 JOB(OCPDBOOT) STATUS=(EXECUTING/ZZZZ),CLASS=STC,", [0m
[0;32m " $HASP890 PRIORITY=15,SYSAFF=(ZZZZ),HOLD=(NONE)", [0m
[0;32m "", [0m
[0;32m "", [0m
[0;32m "Ran 0 \"$D JQ''OCPDBOOT''\" QUIET WAIT" [0m
[0;32m ] [0m
Can see cmd only ended up with single quotes so it works ok
Now using this level of Ansible
ibm.ibm_zos_core 1.8.0
ibm.ibm_zosmf 1.4.1
Ran it today, but it failed
In zOS see
BPXF024I (OMVSKERN) Apr 10 04:18:10 WTZZZZ ansible-ibm.ibm_zos_core.zo
969
s_job_query: Invoked with job_name=OCPDBOOT owner=None job_id=None
IEF450I ZCXPRV18 STEP1 - ABEND=S0C4 U0000 REASON=00000004 970
TIME=04.18.15
IEF450I ZCXPRV19 STEP1 - ABEND=S0C4 U0000 REASON=00000004 971
TIME=04.18.22
BPXF024I (OMVSKERN) Apr 10 04:18:23 WTZZZZ ansible-ibm.ibm_zos_core.zo
972
s_operator: Invoked with cmd=$D JQ''OCPDBOOT'' verbose=False
wait_time_s=1
BPXF024I (OMVSKERN) Apr 10 04:18:23 WTZZZZ ansible-ibm.ibm_zos_core.zo
973
s_operator: Invoked with cmd=$D JQ''OCPDBOOT''
IEA630I OPERATOR ZCXP0000 NOW ACTIVE, SYSTEM=ZZZZ , LU=ZCXPRV1
JQ''OCPDBOOT''
IEE305I JQ''OCPD COMMAND INVALID
IEA631I OPERATOR ZCXP0000 NOW INACTIVE, SYSTEM=ZZZZ , LU=ZCXPRV1
BPXF024I (OMVSKERN) Apr 10 04:18:27 WTZZZZ ansible-ibm.ibm_zos_core.zo
978
s_find: Invoked with patterns=['OCPLDS.OCPDBOOT.ROOT'] resource_type=c
luster age_stamp=creation_date age=None contains=None excludes=None
size=None pds_patterns=None volume=None
from above, see cmd is: $D JQ''OCPDBOOT''
if issue that from SDSF it does fail
Changed ansible, removing one of the double quotes
- name: "{{nodeType }} STC running check using cmd"
delegate_to: zceeZosLpar
ibm.ibm_zos_core.zos_operator:
cmd: "$D JQ'{{ lookup('vars', inventory_hostname).nodeName|upper }}'"
register: bc2Result
after change still fails, see
BPXF024I (OMVSKERN) Apr 10 04:26:01 WTZZZZ ansible-ansible.legacy.copy
079
: Invoked with src=/u/zcxprv1/.ansible/tmp/ansible-tmp-1712737559.8006
68-12027-86366500943514/source dest=/global/zcx/ocpdemo/ocpdboot.prope
rties mode=None follow=False _original_basename=ocp-template-workflow_
variables.openshift.properties checksum=1f188f5ad6981ab2e4d1c5fcf36d9f
86d635d680 backup=False force=True unsafe_writes=False content=NOT_LOG
GING_PARAMETER validate=None directory_mode=None remote_src=None
local_follow=None owner=None group=None seuser=None serole=None
selevel=None setype=None attributes=None
BPXF024I (OMVSKERN) Apr 10 04:26:02 WTZZZZ ansible-ibm.ibm_zos_core.zo
080
s_encode: Invoked with src=/global/zcx/ocpdemo/ocpdboot.properties
encoding={'from': 'ISO8859-1', 'to': 'IBM-1047'} backup=False
backup_compress=False dest=None backup_name=None tmp_hlq=None
BPXF024I (OMVSKERN) Apr 10 04:26:02 WTZZZZ ansible-ibm.ibm_zos_core.zo
081
s_encode: Invoked with src=/global/zcx/ocpdemo/ocpdboot.properties
encoding={'from': 'ISO8859-1', 'to': 'IBM-1047'}
BPXF024I (OMVSKERN) Apr 10 04:26:04 WTZZZZ ansible-ibm.ibm_zos_core.zo
082
s_job_query: Invoked with job_name=OCPDBOOT owner=None job_id=None
BPXF024I (OMVSKERN) Apr 10 04:26:11 WTZZZZ ansible-ibm.ibm_zos_core.zo
083
s_operator: Invoked with cmd=$D JQ'OCPDBOOT' verbose=False
wait_time_s=1
BPXF024I (OMVSKERN) Apr 10 04:26:11 WTZZZZ ansible-ibm.ibm_zos_core.zo
084
s_operator: Invoked with cmd=$D JQ'OCPDBOOT'
IEA630I OPERATOR ZCXP0000 NOW ACTIVE, SYSTEM=ZZZZ , LU=ZCXPRV1
JQ'OCPDBOOT'
IEE305I JQ'OCPDB COMMAND INVALID
IEA631I OPERATOR ZCXP0000 NOW INACTIVE, SYSTEM=ZZZZ , LU=ZCXPRV1
BPXF024I (OMVSKERN) Apr 10 04:26:15 WTZZZZ ansible-ibm.ibm_zos_core.zo
but that cmd entered via sdsf does work
$D JQ'OCPDBOOT'
$HASP890 JOB(OCPDBOOT) 164
$HASP890 JOB(OCPDBOOT) STATUS=(EXECUTING/ZZZZ),CLASS=STC,
$HASP890 PRIORITY=15,SYSAFF=(ZZZZ),HOLD=(NONE)
$HASP890 JOB(OCPDBOOT) 165
$HASP890 JOB(OCPDBOOT) STATUS=(AWAITING HARDCOPY),CLASS=STC,
$HASP890 PRIORITY=1,SYSAFF=(ANY),HOLD=(NONE)
why is it so ?
IBM z/OS Ansible core Version
v1.8.0 (default)
IBM Z Open Automation Utilities
v1.2.5 (default)
IBM Enterprise Python
v3.11.x (default)
ansible-version
v2.16.x (default)
z/OS version
v2.5 (default)
Ansible module
zos_operator
Playbook verbosity output.
No response
Ansible configuration.
No response
Contents of the inventory
No response
Contents of group_vars or host_vars
No response
Demetrios slack conversation: [CSLEJ8VGV/p1712820367812199?thread_ts=1712820340.578559&cid=CSLEJ8VGV]
Came back to read this carefully and looks like is just an escaping issue with the $ symbol, asked the user to try again.
I happened to be looking at something in the same space, originally I thought it might have to do with the single quotes but instead it really had to do with the command prefix $ not being escaped.
I think this issue should be scoped to multiple issues:
- Documentation in the short term, where the docs might instruct users to escape command prefixes, eg
$,#and the doc include some updated samples. One might be to show and example of how to figure out the command prefix the system is using and possibly mention this in documentation as well. For example, this commandD OPDATA,PREFIX(short version I believe isD O) yields:
00- 05.56.00 D OPDATA,PREFIX
05.56.00 IEE603I 05.56.00 OPDATA DISPLAY 634 C
PREFIX OWNER SYSTEM SCOPE REMOVE FAILDSP
$ JES2 EC33017A SYSTEM NO SYSPURGE
MQMA VCA EC33017A SYSTEM NO PURGE
MQMB VCB EC33017A SYSTEM NO PURGE
MQM1 VC1 EC33017A SYSTEM NO PURGE
MQM2 VC2 EC33017A SYSTEM NO PURGE
MQM3 VC3 EC33017A SYSTEM NO PURGE
MQM4 VC4 EC33017A SYSTEM NO PURGE
MQM5 VC5 EC33017A SYSTEM NO PURGE
MQM6 VC6 EC33017A SYSTEM NO PURGE
MQM7 VC7 EC33017A SYSTEM NO PURGE
MQM8 VC8 EC33017A SYSTEM NO PURGE
MQM9 VC9 EC33017A SYSTEM NO PURGE
REXX7A AXR EC33017A SYSPLEX NO PURGE
Showing the JES2 command prefix is $, read more about command prefix in next bullet (2) below.
- A second issue to evaluate the ability to escape command prefixes. This could get a bit complex as JES2 commands use a command prefix, documentation shows it as being a
$but it can be configured to be other special chars, this will need research, you can start by reviewing this section of the doc about Table 2. Command Syntax Conventions
Because your installation establishes the JES2 command identifier, it may be some character other than $. If your installation's command identifier is not specified in your initialization stream by CONCHAR= parameter on the CONDEF statement, the $ is the default.This publication shows the format of a command entered through any console. A command entered through a card reader has a /* (slash asterisk) in card columns 1 and 2 preceding the command identifier.
If the module offers escaping, it will get more complex than the command prefix, research will need to be done for Apostrophes, brackets, parentheses, etc , see Table 2. Command Syntax Conventions
- A follow on to bullet (2) , MVS System commands, see Table 1. System command summary which don't necessarily require a command prefix , see System command formats allow for special characters in the operator commands such as
' # $ & ( ) * + , - . / ¢ < | ! ; ¬ % _ > ? : @ " =, see MVS system commands reference , so additional research would need to be done if the module can escape these as well without altering the commands purpose. I suspect there will be no issue escaping but testing and research will need to be done, I can't think of a MVS system command offhand with a$but one with equals isD M=CPU.
Here is a python successful execution if a similar JES2 command with single quotes.
>>> from zoautil_py import opercmd
>>> from zoautil_py.exceptions import ZOAUException
>>> print(opercmd.execute(command="\$D J\'SSHSTRT\'"))
[ZOAUResponse]
rc: 0
response_format: UTF-8
stdout_response: EC33017A 2024113 12:43:35.00 ISF031I CONSOLE OMVS0000 ACTIVATED
EC33017A 2024113 12:43:35.00 -$D J'SSHSTRT'
EC33017A 2024113 12:43:35.00 $HASP890 JOB(SSHSTRT)
$HASP890 JOB(SSHSTRT) STATUS=(AWAITING HARDCOPY),CLASS=A,
$HASP890 PRIORITY=1,SYSAFF=(ANY),HOLD=(NONE)
stderr_response:
command: opercmd -T 0 -- "\$D J'SSHSTRT' "
This also worked, it depicts not needing to escape single quotes:
print(opercmd.execute(command="\$D J'SSHSTRT'"))
I originally came across this issue
some thoughts...
-
ideally the solution should have a a way to have the cmd passed where no escaping is done, so don't get into these escaping traps, maybe that's not possible in ansible, but this would be the preferred resolution
-
perhaps have a new module - jes2_cmd , where you pass the jes2 cmd without the initial $ and then the code auto adds the $ or as there are other subsystems that you can issue cmds with a leading char - I think MQ can do this - maybe a new module - ssi_cmd - with a parm to set what the initial char and a parm for the rest of the command
-
or change tso_command to pass the in a new parm the command prefix which then get concatenated in front of the cmd supplied
These are all good suggestions @EdwardMcCarthy , the issue is being split into 2 issues, first is to use doc to address and bring awareness.
The second issue (this one) will be to evaluate our options of which some are:
- Consider additional input from a user to let us know if this is a JES or MVS command
- Allow the user to provide the command prefix so we know to escape it
- Since the module supports JES and MVS, create an efficient lookup of which ever is the smaller subset of commands and consider the rest not of that subset, eg, if JES is the smaller list of supported commands, that be efficiently interpreted via a lookup and escape it based on either user prefix input or automatic evaluation, where then the rest are simply MVS commands which won't be escaped. This would mean users don't have to escape anything, but would add overhead to the execution time and possibly regular module maintenance.
Hi
Appreciate it’s a tricky issue.
Re - This would mean users don't have to escape anything, but would add overhead to the execution time and possibly regular module maintenance
I’ not sure I would be too worried about this adding to the execution time. In that says this add a few extra seconds to make or not making escaping decisions, so what, is anyone really going to notice/care ?
The regular maint issue I couldn’t comment on since I don’t write/maintain the code 😉
Re - MVS commands which won't be escaped
There might be zOS cmds like this
F stcid,parm=abc$/%zyx
That is some cmds for products might have a value for a parm that has chars like $, %, etc that may or may not need to be escaped. Offhand I cannot think of such an example right now.
Regards
Edward McCarthy
Senior zStack Technical Specialist Asia/Pacific IBMers Value Dedication to every client's success Innovation that matters - for the company and for the world
Trust and personal responsibility in all relationships Location Code: CAGR
Phone: +61 2 6212 1137 Fax: +61 2 6124 2155
Mobile: +61 411 254 783
Email / Sametime: @.@.>
To infinity and beyond
From: Demetri @.> Sent: Tuesday, April 30, 2024 3:24 AM To: ansible-collections/ibm_zos_core @.> Cc: EDWARD MCCARTHY @.>; Mention @.> Subject: [EXTERNAL] Re: [ansible-collections/ibm_zos_core] [Bug] [zos_operator] Evaluate the ability to escape command prefixes (Issue #1444)
These are all good suggestions @EdwardMcCarthy , the issue is being split into 2 issues, first is to use doc to address and bring awareness. The second issue (this one) will be to evaluate our options of which some are: Consider additional ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. Report Suspicious https://us-phishalarm-ewt.proofpoint.com/EWT/v1/AdhS1Rd-!-XFRWZ5ZeETxVl8SZW8KO4UajSUZK3aTO_xIPkoYdec2D4kUek3ApX2OGVElA7W_Q5yreQLBzAWz9N20ZfSOI7wjnglka314DoxIpQENTZXJgbh1SzZbx7yJiRcB$ ZjQcmQRYFpfptBannerEnd
These are all good suggestions @EdwardMcCarthyhttps://github.com/EdwardMcCarthy , the issue is being split into 2 issues, first is to use doc to address and bring awareness.
The second issue (this one) will be to evaluate our options of which some are:
- Consider additional input from a user to let us know if this is a JES or MVS command
- Allow the user to provide the command prefix so we know to escape it
- Since the module supports JES and MVS, create an efficient lookup of which ever is the smaller subset of commands and consider the rest not of that subset, eg, if JES is the smaller list of supported commands, that be efficiently interpreted via a lookup and escape it based on either user prefix input or automatic evaluation, where then the rest are simply MVS commands which won't be escaped. This would mean users don't have to escape anything, but would add overhead to the execution time and possibly regular module maintenance.
— Reply to this email directly, view it on GitHubhttps://github.com/ansible-collections/ibm_zos_core/issues/1444#issuecomment-2083265138, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APAPVRYKO6PHLRVP3AEYJTDY7Z64NAVCNFSM6AAAAABGDMWFY2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBTGI3DKMJTHA. You are receiving this because you were mentioned.Message ID: @.@.>>
Thanks Edward, we will take into consideration the comment:
MVS commands which won't be escaped
There might be zOS cmds like this
F stcid,parm=abc$/%zyx