Optimizations to be done on fed_benchmark

[jeandut]

Hi,

Thanks for starting the refactoring. Here are a few comments to go even further. I will not mark anything as compulsory given the time constraints, but it would be great to save what will not be done as an issue for future work.

1. Optimization: in fed_benchmark.py, in as much as possible, initialize dataloaders after checking if the experiment needs to be run or not
2. Cutting main in fed_benchmark.py into sub-functions to make it even more readable (it's > 370 lines long at the moment...). I give more actionable suggestions below.
3. Split main into sub-functions pooled_training, local_training, strategy_training. That will make this main much more simpler (but +1 for the single_centric_training which is much better than before).
4. The logic for the tests if the exps are finished or not should be hidden in sub-functions for readability. replace all the len(index_of_interest) < (NUM_CLIENTS + 1) tests to check if experiment is finished by a more explicit function like if check_exp_finished(df, hyperparameters, sname, num_updates) same for all the statements #dealing with edge cases that should not happen --> in sub-functions
5. Lots of variables are assigned very short names, like training_dls for training_dataloaders, s for strategy, m for model... Better to use larger names
6. the logic for filling the hyperparameters' param could also be delegated in sub-functions.
7. conf.py should be renamed into something more explicit, like configuration_utils.py