drizzlepac icon indicating copy to clipboard operation
drizzlepac copied to clipboard

Single visit processing needs to control num_cores parameter

Open stsci-hack opened this issue 5 years ago • 0 comments

The pipeline interface for the single-visit processing, runsinglehap, needs to allow the user or pipeline to control how many CPUs get used by astrodrizzle during processing for each process. This is a feature provided to HSTDP processing for standard pipeline processing to generate the default drizzle products, and would allow them to optimize single-visit processing based on the available resources in the pipeline.

The default value for all single-visit processing has been set to use only 1 CPU to minimize resource usage, especially for very large visits. This would allow the pipeline to specify how many more CPUs to use to process a visit so that larger visits can be processed or reprocessed in a timely manner.

stsci-hack avatar Nov 11 '19 20:11 stsci-hack