[Bug] [zos_copy] Underlying code is creating temporary data sets with HLQ's not permitted to some users

[ddimatos]

Is there an existing issue for this?

• [X] There are no existing issues.

Are the dependencies a supported version?

• [X] The dependencies are supported.

IBM Z Open Automation Utilities

v1.2.5

IBM Enterprise Python

v3.10.x

IBM z/OS Ansible core Version

v1.8.0-beta.1

ansible-version

v2.15.x

z/OS version

v2.5

Ansible module

zos_copy

Bug description

It was reported in issue #1201 through a suggested work around to use zos_copy, the issue here was that a call to dls created temporary data sets which has been corrected in ZOAU 1.3 tracker 10017.

The bulk of the work for this issue is that a new user/segment would need to be created to run the test suite with limited powers, we should use the same user for regression as our current user has to much authority to rule out any issues with RACK limits such as a HLQ.

The problem is, such a change could introduce new test failures not allowing full regression to complete changing the scope of the issue and it size.

Some references that will help:

• ADDSD command
• Steps to defining users to USS
• Optionally but this is not portable is to use one of the extra users we have and limit their powers instead of creating a new one but I would prefer we create new on of our own on any system (of course that requires elevated powers).
• It seems Mike F also encountered similar not long ago have a look at his statck overflow post

Playbook verbosity output.

Taking this from the related issue:

TASK [call_rexx : Copy rexx content pds] ***************************************
task path: /runner/project/roles/call_rexx/tasks/main.yml:22
Thursday 08 February 2024  16:10:58 +0000 (0:00:00.046)       0:00:45.491 ***** 
Thursday 08 February 2024  16:10:58 +0000 (0:00:00.046)       0:00:45.489 ***** 
Read vars_file './playbook_vars.yml'
<cbsk.zos.somecompany.com> ibm_zos_copy SSH transfer method updated from None to sftp.
<cbsk.zos.somecompany.com> ibm_zos_copy: put /tmp/tmphz5t2cog TO /tmp/ansible-zos-copy-payload-D240208-T161058
<cbsk.zos.somecompany.com> SSH: EXEC sshpass -d12 sftp -o BatchMode=no -b - -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/runner/env/tmpmuuq573k"' -o 'User="bakkean"' -o ConnectTimeout=60 -o 'ControlPath="/runner/cp/ca3a767cf3"' '[cbsk.zos.somecompany.com]'
<cbsk.zos.somecompany.com> (0, b'sftp> put /tmp/tmphz5t2cog /tmp/ansible-zos-copy-payload-D240208-T161058\\n', b'')
<cbsk.zos.somecompany.com> ibm_zos_copy return code: 0
<cbsk.zos.somecompany.com> ibm_zos_copy stdout: b'sftp> put /tmp/tmphz5t2cog /tmp/ansible-zos-copy-payload-D240208-T161058\\n'
<cbsk.zos.somecompany.com> ibm_zos_copy stderr: b''
<cbsk.zos.somecompany.com> play context verbosity: 3
<cbsk.zos.somecompany.com> ibm_zos_copy SSH transfer method restored to None
<cbsk.zos.somecompany.com> ibm_zos_copy temp path: /tmp/ansible-zos-copy-payload-D240208-T161058
<cbsk.zos.somecompany.com> ESTABLISH SSH CONNECTION FOR USER: bakkean
<cbsk.zos.somecompany.com> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/runner/env/tmpmuuq573k"' -o 'User="bakkean"' -o ConnectTimeout=60 -o 'ControlPath="/runner/cp/ca3a767cf3"' cbsk.zos.somecompany.com '/bin/sh -c '"'"'echo ~bakkean && sleep 0'"'"''
<cbsk.zos.somecompany.com> (0, b'/u/miscuser/bakkean\\n', b'')
<cbsk.zos.somecompany.com> ESTABLISH SSH CONNECTION FOR USER: bakkean
<cbsk.zos.somecompany.com> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/runner/env/tmpmuuq573k"' -o 'User="bakkean"' -o ConnectTimeout=60 -o 'ControlPath="/runner/cp/ca3a767cf3"' cbsk.zos.somecompany.com '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /u/miscuser/bakkean/.ansible/tmp `"&& mkdir "` echo /u/miscuser/bakkean/.ansible/tmp/ansible-tmp-1707408658.507915-1044-201372063743340 `" && echo ansible-tmp-1707408658.507915-1044-201372063743340="` echo /u/miscuser/bakkean/.ansible/tmp/ansible-tmp-1707408658.507915-1044-201372063743340 `" ) && sleep 0'"'"''
<cbsk.zos.somecompany.com> (0, b'ansible-tmp-1707408658.507915-1044-201372063743340=/u/miscuser/bakkean/.ansible/tmp/ansible-tmp-1707408658.507915-1044-201372063743340\\n', b'')
Using module file /runner/requirements_collections/ansible_collections/ibm/ibm_zos_core/plugins/modules/zos_copy.py
<cbsk.zos.somecompany.com> PUT /home/runner/.ansible/tmp/ansible-local-214co9buyt/tmpsbbo1eoh TO /u/miscuser/bakkean/.ansible/tmp/ansible-tmp-1707408658.507915-1044-201372063743340/AnsiballZ_zos_copy.py
<cbsk.zos.somecompany.com> SSH: EXEC sshpass -d12 sftp -o BatchMode=no -b - -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/runner/env/tmpmuuq573k"' -o 'User="bakkean"' -o ConnectTimeout=60 -o 'ControlPath="/runner/cp/ca3a767cf3"' '[cbsk.zos.somecompany.com]'
<cbsk.zos.somecompany.com> (0, b'sftp> put /home/runner/.ansible/tmp/ansible-local-214co9buyt/tmpsbbo1eoh /u/miscuser/bakkean/.ansible/tmp/ansible-tmp-1707408658.507915-1044-201372063743340/AnsiballZ_zos_copy.py\\n', b'')
<cbsk.zos.somecompany.com> ESTABLISH SSH CONNECTION FOR USER: bakkean
<cbsk.zos.somecompany.com> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/runner/env/tmpmuuq573k"' -o 'User="bakkean"' -o ConnectTimeout=60 -o 'ControlPath="/runner/cp/ca3a767cf3"' cbsk.zos.somecompany.com '/bin/sh -c '"'"'chmod u+x /u/miscuser/bakkean/.ansible/tmp/ansible-tmp-1707408658.507915-1044-201372063743340/ /u/miscuser/bakkean/.ansible/tmp/ansible-tmp-1707408658.507915-1044-201372063743340/AnsiballZ_zos_copy.py && sleep 0'"'"''
<cbsk.zos.somecompany.com> (0, b'', b'')
<cbsk.zos.somecompany.com> ESTABLISH SSH CONNECTION FOR USER: bakkean
<cbsk.zos.somecompany.com> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/runner/env/tmpmuuq573k"' -o 'User="bakkean"' -o ConnectTimeout=60 -o 'ControlPath="/runner/cp/ca3a767cf3"' -tt cbsk.zos.somecompany.com '/bin/sh -c '"'"'LANG=C LIBPATH=/usr/lpp/IBM/zoautil/lib:/usr/local/python/venv_SubSetDist/lib:/lib:/usr/lib:. PATH=/usr/lpp/IBM/zoautil/bin:/usr/local/python/venv_SubSetDist/bin:/bin:/var/bin ZOAU_HOME=/usr/lpp/IBM/zoautil _BPXK_AUTOCVT=ON _CC_LIB_PREFIX=SYS1 _CEE_RUNOPTS='"'"'"'"'"'"'"'"'FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)'"'"'"'"'"'"'"'"' _TAG_REDIR_ERR=txt _TAG_REDIR_IN=txt _TAG_REDIR_OUT=txt /usr/local/python/venv_SubSetDist/bin/python /u/miscuser/bakkean/.ansible/tmp/ansible-tmp-1707408658.507915-1044-201372063743340/AnsiballZ_zos_copy.py && sleep 0'"'"''
<cbsk.zos.somecompany.com> (1, b'\\r\\n{"dest_exists": false, "failed": true, "msg": "Unable to allocate destination data set: BGYSC0239E Error reading from stdin to SYSIN (backing temporary dataset BAKKEAN.P0000321.T0234913.C0000000).\\\\nBGYSC0226E Error allocating DDName SYSPRINT to CONSOLE (backing temporary dataset BAKKEAN.P0000321.T0306570.C0000001).\\\\nBGYSC0220E Error deleting temporary dataset SYSIN=BAKKEAN.P0000321.T0234913.C0000000.\\\\nBGYSC0220E Error deleting temporary dataset SYSPRINT=BAKKEAN.P0000321.T0306570.C0000001.\\\\n", "exception": "  File \\\\"/tmp/ansible_ibm.ibm_zos_core.zos_copy_payload_j8i8gyw2/ansible_ibm.ibm_zos_core.zos_copy_payload.zip/ansible_collections/ibm/ibm_zos_core/plugins/modules/zos_copy.py\\\\", line 2937, in run_module\\\\n  File \\\\"/tmp/ansible_ibm.ibm_zos_core.zos_copy_payload_j8i8gyw2/ansible_ibm.ibm_zos_core.zos_copy_payload.zip/ansible_collections/ibm/ibm_zos_core/plugins/modules/zos_copy.py\\\\", line 2495, in allocate_destination_data_set\\\\n  File \\\\"/usr/local/python/venv_SubSetDist/lib/python3.11/site-packages/zoautil_py/datasets.py\\\\", line 846, in listing\\\\n    raise ZOAUException(response.stderr_response)\\\\n", "invocation": {"module_args": {"content": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", "dest": "MISC.BAKKEAN.SSD.D24039.T161027.IX2.SYSEXEC(REXXRUN)", "tmp_hlq": "", "dest_data_set": {"type": "PDS", "space_primary": 5, "space_secondary": 5, "space_type": "K", "record_format": "VB", "record_length": 255, "block_size": null, "directory_blocks": null, "key_offset": null, "key_length": null, "sms_storage_class": null, "sms_data_class": null, "sms_management_class": null, "name": "MISC.BAKKEAN.SSD.D24039.T161027.IX2.SYSEXEC"}, "is_uss": false, "is_pds": false, "is_src_dir": false, "copy_member": true, "src_member": null, "temp_path": "/tmp/ansible-zos-copy-payload-D240208-T161058", "is_mvs_dest": false, "local_charset": "UTF-8", "is_binary": false, "executable": false, "asa_text": false, "aliases": false, "backup": false, "local_follow": true, "remote_src": false, "ignore_sftp_stderr": false, "validate": false, "use_template": false, "force": false, "force_lock": false, "unsafe_writes": false, "src": null, "encoding": {"from": "UTF-8", "to": "IBM-1047"}, "backup_name": null, "volume": null, "template_parameters": null, "size": null, "mode": null, "owner": null, "group": null, "seuser": null, "serole": null, "selevel": null, "setype": null, "attributes": null, "from_encoding": "UTF-8", "to_encoding": "IBM-1047"}}}\\r\\n', b'Shared connection to cbsk.zos.somecompany.com closed.\\r\\n')
<cbsk.zos.somecompany.com> Failed to connect to the host via ssh: Shared connection to cbsk.zos.somecompany.com closed.
Using module file /runner/requirements_collections/ansible_collections/ibm/ibm_zos_core/plugins/modules/zos_data_set.py
<cbsk.zos.somecompany.com> PUT /home/runner/.ansible/tmp/ansible-local-214co9buyt/tmpgggefczt TO /u/miscuser/bakkean/.ansible/tmp/ansible-tmp-1707408658.507915-1044-201372063743340/AnsiballZ_zos_data_set.py
<cbsk.zos.somecompany.com> SSH: EXEC sshpass -d12 sftp -o BatchMode=no -b - -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/runner/env/tmpmuuq573k"' -o 'User="bakkean"' -o ConnectTimeout=60 -o 'ControlPath="/runner/cp/ca3a767cf3"' '[cbsk.zos.somecompany.com]'
<cbsk.zos.somecompany.com> (0, b'sftp> put /home/runner/.ansible/tmp/ansible-local-214co9buyt/tmpgggefczt /u/miscuser/bakkean/.ansible/tmp/ansible-tmp-1707408658.507915-1044-201372063743340/AnsiballZ_zos_data_set.py\\n', b'')
<cbsk.zos.somecompany.com> ESTABLISH SSH CONNECTION FOR USER: bakkean
<cbsk.zos.somecompany.com> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/runner/env/tmpmuuq573k"' -o 'User="bakkean"' -o ConnectTimeout=60 -o 'ControlPath="/runner/cp/ca3a767cf3"' cbsk.zos.somecompany.com '/bin/sh -c '"'"'chmod u+x /u/miscuser/bakkean/.ansible/tmp/ansible-tmp-1707408658.507915-1044-201372063743340/ /u/miscuser/bakkean/.ansible/tmp/ansible-tmp-1707408658.507915-1044-201372063743340/AnsiballZ_zos_data_set.py && sleep 0'"'"''
<cbsk.zos.somecompany.com> (0, b'', b'')
<cbsk.zos.somecompany.com> ESTABLISH SSH CONNECTION FOR USER: bakkean
<cbsk.zos.somecompany.com> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/runner/env/tmpmuuq573k"' -o 'User="bakkean"' -o ConnectTimeout=60 -o 'ControlPath="/runner/cp/ca3a767cf3"' -tt cbsk.zos.somecompany.com '/bin/sh -c '"'"'LANG=C LIBPATH=/usr/lpp/IBM/zoautil/lib:/usr/local/python/venv_SubSetDist/lib:/lib:/usr/lib:. PATH=/usr/lpp/IBM/zoautil/bin:/usr/local/python/venv_SubSetDist/bin:/bin:/var/bin ZOAU_HOME=/usr/lpp/IBM/zoautil _BPXK_AUTOCVT=ON _CC_LIB_PREFIX=SYS1 _CEE_RUNOPTS='"'"'"'"'"'"'"'"'FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)'"'"'"'"'"'"'"'"' _TAG_REDIR_ERR=txt _TAG_REDIR_IN=txt _TAG_REDIR_OUT=txt /usr/local/python/venv_SubSetDist/bin/python /u/miscuser/bakkean/.ansible/tmp/ansible-tmp-1707408658.507915-1044-201372063743340/AnsiballZ_zos_data_set.py && sleep 0'"'"''
<cbsk.zos.somecompany.com> (0, b'\\r\\n{"changed": false, "message": "", "names": ["MISC.BAKKEAN.SSD.D24039.T161027.IX2.SYSEXEC(REXXRUN)"], "invocation": {"module_args": {"name": "MISC.BAKKEAN.SSD.D24039.T161027.IX2.SYSEXEC(REXXRUN)", "state": "absent", "type": "MEMBER", "space_type": "M", "space_primary": 5, "space_secondary": 3, "record_format": "FB", "replace": false, "force": false, "batch": null, "sms_management_class": null, "sms_storage_class": null, "sms_data_class": null, "block_size": null, "directory_blocks": null, "record_length": null, "key_offset": null, "key_length": null, "volumes": null, "tmp_hlq": null}}}\\r\\n', b'Shared connection to cbsk.zos.somecompany.com closed.\\r\\n')
fatal: [localhost -> CBSK(cbsk.zos.somecompany.com)]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "content": "/* REXX */        \\nvar = \\"'MISC.BAKKEAN.XYZAAA' VOLUME(GCKD03) DIRECTORY SMSINFO\\" \\nrc=listdsi(var )        \\nsay rc||\\"|\\"||,\\n  SYSDSNAME||\\"|\\"||,\\n  SYSVOLUME||\\"|\\"||,\\n  SYSUNIT||\\"|\\"||,\\n  SYSDSORG||\\"|\\"||,\\n  SYSRECFM||\\"|\\"||,\\n  SYSLRECL||\\"|\\"||,\\n  SYSBLKSIZE||\\"|\\"||,\\n  SYSKEYLEN||\\"|\\"||,\\n  SYSALLOC||\\"|\\"||,\\n  SYSUSED||\\"|\\"||,\\n  SYSUSEDPAGES||\\"|\\"||,\\n  SYSPRIMARY||\\"|\\"||,\\n  SYSSECONDS||\\"|\\"||,\\n  SYSUNITS||\\"|\\"||,\\n  SYSEXTENTS||\\"|\\"||,\\n  SYSUSEDEXTENTS||\\"|\\"||,\\n  SYSCREATE||\\"|\\"||,\\n  SYSREFDATE||\\"|\\"||,\\n  SYSEXDATE||\\"|\\"||,\\n  SYSPASSWORD||\\"|\\"||,\\n  SYSRACFA||\\"|\\"||,\\n  SYSUPDATED||\\"|\\"||,\\n  SYSTRKSCYL||\\"|\\"||,\\n  SYSBLKSTRK||\\"|\\"||,\\n  SYSADIRBLK||\\"|\\"||,\\n  SYSUDIRBLK||\\"|\\"||,\\n  SYSMEMBERS||\\"|\\"||,\\n  SYSREASON||\\"|\\"||,\\n  SYSMSGLVL1||\\"|\\"||,\\n  SYSMSGLVL2||\\"|\\"||,\\n  SYSDSSMS||\\"|\\"||,\\n  SYSDATACLASS||\\"|\\"||,\\n  SYSSTORCLASS||\\"|\\"||,\\n  SYSMGMTCLASS||\\"|\\"||,\\n  SYSSEQDSNTYPE||\\"|\\"||,\\n  SYSEATTR||\\"|\\"||,\\n  SYSEADSCB||\\"|\\"||,\\n  SYSALLOCPAGES||\\"|\\"||,\\n  SYSUSEDPERCENT||\\"|\\"||,\\n  SYSNUMVOLS||\\"|\\"||,\\n  SYSVOLUMES||\\"|\\"||,\\n  SYSCREATETIME||\\"|\\"||,\\n  SYSCREATESTEP||\\"|\\"||,\\n  SYSCREATEJOB||\\"|\\"||,\\n  SYSDSVERSION||\\"|\\"||,\\n  SYSENCRYPT||\\"|\\"||,\\n  SYSKEYLABEL||\\"|\\"||,\\n  SYSMAXGENS||\\"|\\" /* the last | after SYSMAXGENS is intentional, to be able to remove  READ and END easily from output  */\\n",
            "dest": "MISC.BAKKEAN.SSD.D24039.T161027.IX2.SYSEXEC(REXXRUN)",
            "dest_data_set": {
                "record_format": "VB",
                "record_length": 255,
                "space_primary": 5,
                "space_secondary": 5,
                "space_type": "K",
                "type": "PDS"
            },
            "tmp_hlq": ""
        }
    },
    "msg": "Unable to allocate destination data set: BGYSC0239E Error reading from stdin to SYSIN (backing temporary dataset BAKKEAN.P0000321.T0234913.C0000000).\\nBGYSC0226E Error allocating DDName SYSPRINT to CONSOLE (backing temporary dataset BAKKEAN.P0000321.T0306570.C0000001).\\nBGYSC0220E Error deleting temporary dataset SYSIN=BAKKEAN.P0000321.T0234913.C0000000.\\nBGYSC0220E Error deleting temporary dataset SYSPRINT=BAKKEAN.P0000321.T0306570.C0000001.\\n",
    "rc": null,
    "stderr": null,
    "stderr_lines": null,
    "stdout": null,
    "stdout_lines": null
}

Ansible configuration.

N/A

Contents of the inventory

N/A

Contents of group_vars or host_vars

N/A

[ddimatos]

Although there is some validation to do for zoau 1.3 , this issue is more about creating the correct test framework updates to test hlq and tmp_hlq.

[fernandofloresg]

Demetri recommended to ask Frank D.