Parallelize make run using SLURM

[mikebentley15]

As seen in this blog post:

https://plindenbaum.blogspot.com/2014/09/parallelizing-gnu-make-4-in-slurm.html

for GNU Make version 4.0 and later, we can actually execute all Makefile recipes as slurm jobs. This is pretty cool. We could add a --slurm flag to flit make to take advantage of that. The only thing of note is that we may want to look at overriding the MAKE variable so that each submake invocation has its own set of jobs. For example if -j 72 is used on flit make, then each node that runs a submake will also have 72 jobs running. That's a lot of parallelism, for not too much effort.

To make use of this, we would overwrite the following environment variables, and maybe a few others too:

• SHELL: this would become srun
• .SHELLARGS: this would be something like -N1 -p <something> ...
• MAKE: for sub-invocations of make, probably set to make SHELL=$(SHELL) .SHELLARGS=$(.SHELLARGS) -j $(JOBS) or something like that. This requires investigation to see if it would work