flux-core icon indicating copy to clipboard operation
flux-core copied to clipboard

resource: constant load on large system

Open grondo opened this issue 8 months ago • 4 comments

This was briefly discussed in the dev meeting last week, but there's a somewhat constant load (40% of a CPU) on the rank 0 broker on a large system attributed to the resource module here:

-   56.31%     0.00%  resource         libc-2.28.so           [.] __GI___clone ◆
     __GI___clone (inlined)                                                    ▒
     start_thread                                                              ▒
     module_thread                                                             ▒
     mod_main                                                                  ▒
     flux_reactor_run                                                          ▒
     ev_run                                                                    ▒
   - ev_run                                                                    ▒
      - 56.31% ev_invoke_pending                                               ▒
         - 56.29% sched_status_continuation                                    ▒
            - 56.27% prepare_sched_status_payload (inlined)                    ▒
               + 29.76% get_all (inlined)                                      ▒
               + 14.34% create_rlist (inlined)                                 ▒
               + 6.49% get_down (inlined)                                      ▒
               + 2.09% update_properties_json (inlined)                        ▒
               + 1.83% rlist_destroy                                           ▒
               + 1.39% get_empty_set (inlined)                                

Probably not critical, but caching the "all" resource set seems like a simple approach that would have a large impact.

grondo avatar May 28 '24 14:05 grondo