Extend zowe files copy data-set command to allow for copying across LPARs / z/OSMF instances

[MikeBauerCA]

Currently, the zowe files copy data-set command is useful for easily copying data-sets on the mainframe. However, you may need to copy a data-set across LPARs. Today, you would need to download then upload using the local workstation as an intermediary. This approach may be okay "behind the scenes" but should be transparent to the end user.

[JTonda]

When copying datasets, ensure that the DCB information is retained.

Note: Check to see if target dataset exists. If exists, check to see if DCB info matches that of source. If not, stop. If yes, then we would proceed with copy (download upload).

If target dataset does not exist, then create (with same DCB info as source) and download upload. If we fail on upload and we had created the target, we would clean up that target.

Also may need to consider the .zosattributes file. May want to see if direct file transfer is possible instead of using local workstation as intermediary. If we use the workstation as an intermediary, investigate file streaming. Investigate using binary/record download upload. Discussion ongoing.

[awharn]

Suggestion if we do need to use a local workstation as an intermediary (this is just a suggestion in the dark, might not be the best idea):

1. Read the DCB
2. Attempt to allocate the new dataset with that DCB
3. Stream in record format

[github-actions[bot]]

Thank you for raising this enhancement request. The community has 90 days to vote on it. If the enhancement receives at least 5 upvotes, it is added to our development backlog. If it receives fewer votes, the issue is closed.

[awharn]

After doing some review, I have created a flowchart to step through the steps in a flow that I think would be beneficial.

Within the first 2 REST requests, we would know that the source dataset exists and both sessions are valid and authenticated, failing early for most major errors.

If we are going to be replacing a dataset, I am of the opinion that, if the user decides the dataset should be overwritten, the conflicting target dataset should be deleted and reallocated to make sure the attributes match the source (or we could check the attributes and skip if they match). That way we minimize errors we might run into if the attributes are not correct for the source dataset to be uploaded.

If it is a Member, we need to check that the member itself does not exist. Whether or not we overwrite the member, we should NOT reallocate the dataset in that case.

In all cases, List Source Dataset is to grab the source dataset attributes. We don't need to do that in the case of copying a member across LPARs UNLESS we need to allocate the PDS(E).

flowchart TD
    %% Main actions
    getsource[Get Source Contents]
    listtarget[List Target Dataset]
    dsexists{{Target <br> Dataset <br> Exists?}}
    listsource[List Source Dataset]
    allocate[Allocate Target Dataset]
    upload[Upload Dataset/Member Contents]

    %% Overwrite Actions (main)
    targetmember{{Target Is A Member?}}
    abort[Abort Operations]
    
    %% Overwrite Actions (DS)
    dsprompt{{Prompt <br> for <br> Overwrite}}
    dslistsource[List Source Dataset]
    dsdeletetarget[Delete Target Dataset]

    %% Overwrite Actions (member)
    memberget[Get Target Member]
    memberexists{{Target <br> Member <br> Exists}}
    memberprompt{{Prompt <br> For <br> Overwrite}}

    %% Failure actions
    failnods[Fail - No Dataset]
    failbadcreds[Fail - Bad Credentials]

    %% Main connections
    getsource ----> listtarget
    listtarget ----> dsexists
    dsexists -- No --> listsource
    dsexists -- Yes --> targetmember
    listsource ----> allocate
    allocate ----> upload

    %% Overwrite connections (main)
    targetmember -- No --> dsprompt
    targetmember -- Yes --> memberget

    %% Overwrite connections (DS)
    dsprompt -- Ok --> dslistsource
    dsprompt -- No --> abort
    dslistsource ----> dsdeletetarget
    dsdeletetarget ----> allocate

    %% Overwrite connections (member)
    memberget ----> memberexists
    memberexists -- No --> upload
    memberexists -- Yes --> memberprompt
    memberprompt -- No --> abort
    memberprompt -- Ok --> upload

    %% Failure connections
    getsource -- DS missing --> failnods
    getsource -- Auth failure --> failbadcreds
    listtarget -- Auth failure --> failbadcreds