ibm_zos_core
ibm_zos_core copied to clipboard
ansible-collections
[Bug] [zos_job_submit] When using zos_job_submit templating from within a role, the variable was not popluated.
Is there an existing issue for this?
- [X] There are no existing issues.
Bug description
Note that this was created using IBM z/OS Core 1.7.0 (not 1.9.0 which is the latest on time) purposely because with 1.9.0 the zos_job_submit action plugin uses a deepcopy call and breaks, never getting this far where a template is created. When working on this, I would suggest that isues #1251 and #1498 be reviewed.
Also note that this only happens when the role is used with loop , if you remove the loop and define a variable, it correctly is transferred to USS.
Since the template is not logged until issue #1231 is complete, I wrote a small while loop watching for the template and saw the JCL had not been substituted the variable.
while(true); do ls -la tmp*;cat tmp*;sleep 1;done
cat: tmp*: EDC5129I No such file or directory.
ls: FSUM6785 File or directory "tmp*" is not found
cat: tmp*: EDC5129I No such file or directory.
-rw------- 1 BPXROOT OMVSGRP 0 May 4 16:31 tmpngtf2bki
-rw------- 1 BPXROOT OMVSGRP 277 May 4 16:31 tmpngtf2bki
/LISTC JOB (T043JM,JM00,1,0,0,0),'LISTC',
// MSGCLASS=H,MSGLEVEL=1,NOTIFY=&SYSUID
//*-----------------------------------------------------
//*
//GO EXEC PGM=IDCAMS,REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
LISTCAT ENTRIES('{{ item.dataset_name }}')
Notice below, the role did see the Data Set name, so its not an issue of the data set name not getting propagated.
"msg": "dataset_name: ANSIBLE.DATASET.JCL"
Error:
(venv-2.16) ansible-playbook -i inventory zos_job_submit_loop.yml
PLAY [zvm] *****************************************************************************************************************************************************************************************************************
TASK [sumit JCL in a role with loop] ***************************************************************************************************************************************************************************************
TASK [jcl_test_role : debug] ***********************************************************************************************************************************************************************************************
ok: [zvm] => {
"msg": "dataset_name: ANSIBLE.DATASET.JCL"
}
TASK [jcl_test_role : submit JCL2 in a role] *******************************************************************************************************************************************************************************
fatal: [zvm]: FAILED! => {"changed": false, "checksum": "d24fd16105a55bf082dc3cdff588ec12e731bf80", "dest": "/tmp/ansible.p89qs83u", "duration": 0, "gid": 1, "group": "OMVSGRP", "job_id": null, "md5sum": "9aef3827d32f7d41babfa5ff1c1697dc", "mode": "0600", "msg": "The JCL has been submitted /Users/ddimatos/git/playbooks/roles/jcl_test_role/templates/listc_template.jcl and no job id was returned within the allocated time of 10 seconds. Consider using module zos_job_query to poll for a long running jobs or increasing the value for `wait_times_s`.", "owner": "BPXROOT", "size": 277, "src": "/.ansible/tmp/ansible-tmp-1714865444.560121-75773-171782903963966/source", "state": "file", "stderr": "Unable to submit Job: {'rc': 8, 'response_format': 'UTF-8', 'stdout_response': '', 'stderr_response': 'BGYSC4003E Job submission failed.\\n', 'command': 'jsub -f \"/tmp/tmpngtf2bki\"'}", "stderr_lines": ["Unable to submit Job: {'rc': 8, 'response_format': 'UTF-8', 'stdout_response': '', 'stderr_response': 'BGYSC4003E Job submission failed.\\n', 'command': 'jsub -f \"/tmp/tmpngtf2bki\"'}"], "uid": 0}
Playbook: playbooks/zos_job_submit_loop.yml
---
- hosts: zvm
collections:
- ibm.ibm_zos_core
gather_facts: false
vars:
dataset_list:
- dataset_name : "SYS1.PROCLIB"
environment: "{{ environment_vars }}"
tasks:
# Test role
- name: sumit JCL in a role with loop
vars:
dataset_name: "{{ item.dataset_name }}"
include_role:
name: jcl_test_role
loop: "{{ dataset_list }}"
The template: playbooks/roles/jcl_test_role/templates/listc_template.jcl
/LISTC JOB (T043JM,JM00,1,0,0,0),'LISTC',
// MSGCLASS=A,MSGLEVEL=1,NOTIFY=&SYSUID
//*-----------------------------------------------------
//*
//GO EXEC PGM=IDCAMS,REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
LISTCAT ENTRIES('{{ dataset_name }}')
/*
The role: playbooks/roles/jcl_test_role/tasks/main.yml
---
# tasks file for jcl_test_role
- name: debug
debug:
msg: "dataset_name: {{ dataset_name }}"
- name: submit JCL2 in a role
ibm.ibm_zos_core.zos_job_submit:
src: "{{ playbook_dir }}/roles/{{ role_name }}/templates/listc_template.jcl"
location: LOCAL
encoding:
from: ISO8859-1
to: IBM-1047
use_template: true
To create a role:
mkdir roles
cd roles/
ansible-galaxy init jcl_test_role
Then add the contents of the role (above) to jcl_test_role/tasks/main.yml
I also tried this with IBM z/OS Core 1.7.0 when use of deep copy was not used copy_task = copy.deepcopy(self._task).
I assume that the serialization from the deep copy's pickling is failing. This relates to issue #1251 , its possible this issue could be resolved with #1251 but note when testing version 1.7.0, although no deep copy error occurred the template variable was not inserted into the JCL , I will open another issue for that.
IBM z/OS Ansible core Version
v1.7.0
IBM Z Open Automation Utilities
v1.2.5 (default)
IBM Enterprise Python
v3.10.x
ansible-version
v2.16.x (default)
z/OS version
v2.5 (default)
Ansible module
No response
Playbook verbosity output.
No response
Ansible configuration.
No response
Contents of the inventory
No response
Contents of group_vars or host_vars
No response
@rexemin does this one relate to #940 , they seem different but play in the same space, maybe we should have an epic to group these issues all together for one person to work on?
Judging from the way the variable inside the template went from dataset_name to item.dataset_name, it seems the template is substituting correctly, but it lacks the code to recursively substitute until we have the final value. So I would say this is the same issue as #1071, but now we're seeing it also affects variables inside a loop.
Setting the priority to the same level as the related issue, and giving it 1 point at this time, mostly because this item should be testing the enhancement made for #1071.
We are facing a issue that i think is related, on zos_copy template do not resolve variables from loop... We can use any other variable but the ones that should get resolved from loop like item, indexes, are empty in the template results.
Here is one way to replicate...
main playbook:
---
- name: Testing copy template
hosts: lpar
gather_facts: true
vars:
broker_name:
- AB
- CD
key_password: testpsw
environment: "{{ environment_vars }}"
tasks:
- name: Testing copy module
ansible.builtin.include_tasks: task_broker.yml
loop: "{{ broker_name }}"
loop_control:
index_var: broker_index
- name: Testing copy module 2
ibm.ibm_zos_core.zos_copy:
dest: "/home/testbroker2{{ item }}.jcl"
src: "{{ playbook_dir }}/broker2.j2"
use_template: true
force: true
loop: "{{ broker_name }}"
task_broker.yml:
---
- name: Assign variable for templating
ansible.builtin.set_fact:
broker_name_tmp: "{{ item }}"
- name: Testing copy module
ibm.ibm_zos_core.zos_copy:
dest: "/home/testbroker{{ broker_name_tmp }}.jcl"
src: "{{ playbook_dir }}/broker.j2"
use_template: true
force: true
broker.j2:
//$BROCONF JOB (FTM99),
// 'FTM DPL',
// USER={{broker_user}},
// CLASS=I,
// REGION=1400M,
// MEMLIMIT=1800M,
// COND=(4,LT)
//*****************************************************************
//*** Configure keystore
//******************************************************************
/*JOBPARM SYSAFF={{ target_host | upper }}5
//REPORT1 EXEC PGM=IKJEFT1B
//STDENV DD PATHOPTS=(ORDONLY),
// PATH='/u/ib{{broker_name_tmp}}u/ENVFILE'
//STDOUT DD SYSOUT=*
//STDERR DD SYSOUT=*
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
BPXBATSL PGM /usr/lpp/mqsi/V10R0M0/bin/-
mqsichangeproperties IB{{broker_name_tmp| upper}} -
-b httplistener -
-o HTTPSConnector -
-n keystoreFile -
broker2.j2
//$BROCONF JOB (FTM99),
// 'FTM DPL',
// USER={{broker_user}},
// CLASS=I,
// REGION=1400M,
// MEMLIMIT=1800M,
// COND=(4,LT)
//*****************************************************************
//*** Configure keystore
//******************************************************************
/*JOBPARM SYSAFF={{ target_host | upper }}5
//REPORT1 EXEC PGM=IKJEFT1B
//STDENV DD PATHOPTS=(ORDONLY),
// PATH='/u/ib{{item}}u/ENVFILE'
//STDOUT DD SYSOUT=*
//STDERR DD SYSOUT=*
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
BPXBATSL PGM /usr/lpp/mqsi/V10R0M0/bin/-
mqsichangeproperties IB{{item| upper}} -
-b httplistener -
-o HTTPSConnector -
-n keystoreFile -
if we try to access any variablefrom playbook it gets resolved, variables from loop doesnt get resolved, also we noticed if you dont define a variable, if its missing it doesnt give any error just goes on with empty value...
We tried it with zos collections 1.8 and 1.10, we get same results on both
Thanks @billpereira , we have a few related issues here, we need to group them under an epic and put it in plan, for now if you pan over to the right of an issue, you should be able to see how our item is planned, expand out the projects section and see when its gets put in plan. I did this after I signed out of GH and was able to see it, so I believe everyone can see it. Additional recreates are very helpful, thank you.