Enable transfers of archived data to Niagara from non-HPSS-connected systems

[DavidHuber-NOAA]

Description

As part of the archive (earc, arch) jobs, an option to push data to Niagara from Globus Endpoint machines would allow users to stage data for archive to HPSS. This feature would require that LOCAL_ARCH="YES" be set. This option would be useful for Orion, S4, and eventually Hercules.

Acceptance Criteria (Definition of Done)

Data is pushed to HPSS from Orion and verified against the original .tar files created on Orion.

(Optional): Suggest A Solution

The globus transfers could be made part of the *arc j-jobs and added after each successful creation of a tar file.

At the end of each *arc script, an HPSS script would be generated and globus transferred to Niagara containing a list of all .tar files that were sent. A crontab entry could then be placed on Niagara to look for and execute these scripts and report any errors via email.

@erinjones2 developed scripts that worked on Orion for the feature/ops-orion branch. I plan on adapting these to the purpose.