Proposal for Homogenizing Computing Element Interfaces in DIRAC

[aldbr]

The diversity and complexity of Computing Elements (CEs) in DIRAC have significantly increased, leading to a somewhat disorganized system. We have observed that the current structure, which broadly categorizes CEs into Remote (e.g. ARC, HTCondor, SSH, ...), Inner (e.g. InProcess, Singularity), and Inner Pool (e.g. Pool) types, is becoming increasingly unwieldy. This situation necessitates a more streamlined and unified approach to enhance maintainability and usability.

I propose a unified interface approach for all types of CEs. The core idea is to make these interfaces homogeneous, thus simplifying the interaction and management of various CE types.

Key aspects of the proposal include:

• Submission status returns: For Inner CEs (InProcess, Singularity, Pool), the submission method should return the status of the submission but not the payload's exit code directly.
• Introduction of getJobStatus() method: Implement a getJobStatus() method for Inner CEs. This method would retrieve the job's status from taskResults similarly to how it's currently handled by Remote CEs. This approach ensures consistency across different CE types and simplifies status retrieval mechanisms.
• Enhanced Maintainability: By streamlining the interface, the maintenance and development of CEs within DIRAC would become more manageable, allowing for easier updates and potential expansions in the future.

The proposal was introduced in https://github.com/DIRACGrid/DIRAC/pull/6970 but not followed up.