hpc-intro
hpc-intro copied to clipboard
carpentries-incubator
Scheduler Concepts
Would it be helpful/possible to create a list of key concepts (scheduler agnostic) that should be discussed in the scheduler lesson? Syntax specific to whatever scheduler is being used could then be filled in around those concepts. I pulled a list of ideas from the current SLURM lesson as a starting point:
- Definition of a job and batch processing
- Submitting a job to the scheduler
- Passing options to the scheduler
- Changing a job's name
- Send an email once the job completes
- Requesting resources on a compute node
- Log files/job status
- Wall times
- Cancelling/deleting a job
Agree 100%. I think the scheduler part needs a fair bit of work, see #31.
I would have changing a job's name + sending an email as callouts (not required) or extras.
I added your comments into #40. Please have a look at it and comment. This should then serve as a guide on how to move forward in the coming days.
Scheduler policy vary from center to center. May just want to have a broad overview. Some facilities prioritize large jobs. Many also have a notion of fair share. It is unlikely a detailed analysis can be done. It might be better to ask participants to think about and then discuss what they think would be a suitable scheduling policy to be used at an HPC facility of importance to them. The current example also uses a program fastqc that may not be installed on every hpc facility. It may be helpful to use a script that is portable
Our afternoon meeting suggested purging fastqc for pedagogical reasons, which eliminates the difficulty of not having fastqc pre-installed. Note that it's a simple program which is easy to copy from GitHub and compile locally.
