FATE icon indicating copy to clipboard operation
FATE copied to clipboard

Provide an "estimated time cost" for example cases

Open ZiyueXu77 opened this issue 2 years ago • 6 comments

Is your feature request related to a problem? Please describe. I am running a testing example with standalone docker 1.11.3: "https://github.com/FederatedAI/FATE/blob/master/examples/benchmark_quality/hetero_sbt/fate-sbt.py" with basic binary setting "https://github.com/FederatedAI/FATE/blob/master/examples/benchmark_quality/hetero_sbt/fate_sbt_binary.yaml".

The training time is about 11 min. I am not sure if such time cost is expected / reasonable, or if I made any mistakes during my experiment setting up. This is mainly because if I perform centralized xgboost training, the training will finish within seconds.

Describe the solution you'd like I hope an "expected running time" can be added to each of the examples (under certain hardware/system configurations), so that users will roughly know what to expect for training time.

Describe alternatives you've considered Some instructions on the common time consumption for different computations and tricks that can be added for speed improvement

Additional context Add any other context or screenshots about the feature request here.

ZiyueXu77 avatar Nov 10 '23 01:11 ZiyueXu77

Hi! The training time of the example case on your machine is in expectation. Federated sbt training requires Paillier cipher computations which cost much time than centralized plaintext computing.

talkingwallace avatar Nov 10 '23 03:11 talkingwallace

You can refer to this paper: https://arxiv.org/abs/2110.10927, to have a general sense of training time cost.

talkingwallace avatar Nov 10 '23 03:11 talkingwallace

Thanks! appreciate your input

ZiyueXu77 avatar Nov 10 '23 15:11 ZiyueXu77

The default (not providing "Paillier" to encrypt_param) is without performing any encryption right?

ZiyueXu77 avatar Nov 10 '23 16:11 ZiyueXu77

The default (not providing "Paillier" to encrypt_param) is without performing any encryption right?

No, default is always Paillier, encryption is compulsory

talkingwallace avatar Nov 15 '23 02:11 talkingwallace

ah I see, so there is no option to "turn-off" the encryption (just to check the "base" time cost)?

ZiyueXu77 avatar Nov 15 '23 15:11 ZiyueXu77