FATE icon indicating copy to clipboard operation
FATE copied to clipboard

资源配置参数 task_cores 和 computing_partitions 的含义

Open hust-suwb opened this issue 1 year ago • 1 comments
trafficstars

从 https://fate-flow.readthedocs.io/en/v1.11.1/fate_flow_resource_management/ 文档上看,其实我只需要配置 task_cores 就能设定任务使用的cpu资源,然而我实测下来发现:

a. task_cores = 12 & computing_partitions = 12 b. task_cores = 24 & computing_partitions = 12 c. task_cores = 24 & computing_partitions = 24

任务耗时 a 约等于 b 大于 c。且从机器监控上看,cpu利用率也是 a 约等于 b 小于 c。

也就是说,仅靠 task_cores 的配置是不够的,还必须同步去配置 computing_partitions 的值才能达到最佳的效果。这个信息与文档中的表述似乎不太一致,不确定到底是为什么。求指教。

总Core:40 Upload指定的分区数:8 OS:CentOS7 FATE:1.11.1 配置方式: "job_parameters":{ "common":{ "computing_partitions":12, "task_cores":24 } }

hust-suwb avatar Jul 23 '24 03:07 hust-suwb

computing_partitions应该是计算的时候数据的分区数,而task_cores对应的是使用多少个cpu进程去处理这个数据,每个cpu进程每次会拿一个分区的数据去进行计算,直到处理完所有分区。所以task_cores应该是<=computing_partitions的。当不设置的时候,task_cores应该会等于computing_partitions。

mgqa34 avatar Jul 23 '24 09:07 mgqa34

This issue has been marked as stale because it has been open for 365 days with no activity. If this issue is still relevant or if there is new information, please feel free to update or reopen it.

github-actions[bot] avatar Jul 24 '25 03:07 github-actions[bot]

This issue was closed because it has been inactive for 1 days since being marked as stale. If this issue is still relevant or if there is new information, please feel free to update or reopen it.

github-actions[bot] avatar Jul 26 '25 03:07 github-actions[bot]