[9.0] feat: SSHCE export inputs/import outputs as a JSON file to handle more jobs in parallel
In LHCb, we would like to submit a very large number of jobs in parallel via the SSHCE, and we are generally limited to about 300 jobs per cycle because the stamps are generated by the SSHCE and sent through SSH to a batch system. Problem: the line sent is way too long and cannot be executed.
Note: I have not added too many checks for failures for now, I keep it simple and will give it a try in LHCb to see how it works.
BEGINRELEASENOTES *Resources CHANGE: SSHCE export inputs/import outputs as a JSON file instead of using the stdout/stderr ENDRELEASENOTES
Monitor jobs by batch of 1000 (or more) instead of 100
Going through this I discovered a few old problems with the SSHCE, as well as a minor security issue that I will solve in another PR, but these changes are working. I think it's also a good opportunity to finish the scripts I am making for testing the CEs, and finish https://github.com/DIRACGrid/DIRAC/pull/7703