Feature: Storage plugin

[lecorguille]

Description

Definitely, interesting in Coldfront to manage our HPC SLURM insfrastructure.

We are wondering how Coldfront can interact with a storage filesystem as it does with slurm. Through our current account manager, a PI can request what we call a project with CPU hours and storage. As soon as the admin approuve the request are created:

• a slurm account (access to partition, qos)
• a project folder
• a quota on the project folder

The application provide some kind of template scripts where you can adapt the command to fit with the specific filesystem commands.

Is any existing plugin for storage which can address this?

Thanks

Component

Resources

Additional information

No response

[dsajdak]

@lecorguille there is nothing built by the ColdFront team to address this yet. We're not aware of any community plug-ins either. Since all of this is happening outside of ColdFront and you already have scripts that do all this, you could potentially create a Django plug-in similar to the Slurm or FreeIPA plugin and use the Django queue to run that task on a given time frame or when the allocation is activated by an admin. Since these types of workflows differ from center to center, we've held off on implementing this but I'm going to leave this feature request open because many people have asked us about it in the past. When we have the development time it's something we can look into.

[jrlagrone]

@lecorguille we're also implementing some version of this "soon."

We're still discussing what everything will look like. We are planning on storage and compute allocations being separate, with most users having some "personal" storage allocation by default. Our solution(s) will be targeting Lustre project based storage quotas with access/permissions controlled mostly with ACLs.

Since it's not implemented yet, I don't want to promise anything, but we are interested in submitting PRs when we can.