SourceXtractorPlusPlus
SourceXtractorPlusPlus copied to clipboard
Investigate SE++ scaling with multi-threading
I ran some Euclid test data with single image fitting (3 bands, ~4 images per band per pixels) using different number of cores to test the scaling of the multi-threading. This investigation is similar to #411, but some more care was taken to only use physical cores (no hyperthreading). The system information is:
- This system has 2 physical CPUs, and 128 logical CPUs.
- For every physical CPU there are 32 cores.
- The CPU is a AMD EPYC 7513 32-Core Processor with 512 KB cache
There are 64 physical cores. I ran it with different # of physical cores which results to (the basic data is below):
Note that I did use hyperthreading for the last data point wit 124 cores. I repeated the measurements for many core numbers (with days in between the runs) but finding virtually identical run times.
# 52197 objects # ncore time_nohyper time_nh 4 38:50:07 38.835 8 20:07:38 20.127 8 20:13:18 20.222 12 13:21:53 13.365 16 10:30:01 10.500 16 10:31:01 10.500 20 09:56:07 9.935 24 08:18:02 8.300 24 08:16:05 8.268 28 06:29:01 6.484 32 05:41:24 5.690 32 05:42:07 5.702 36 05:19:22 5.323 40 05:08:39 5.144 40 05:05:54 5.098 44 04:25:53 4.432 48 04:01:26 4.023 48 04:04:29 4.074 52 03:53:10 3.886 56 03:49:48 3.830 56 03:49:02 3.817 60 03:27:04 3.451 64 03:24:02 3.400 64 03:24:32 3.409 128 03:00:22 3.006