FedScale

FedScale copied to clipboard

Issues on FedScale and Oort: (1) widely promotes its so-called advantages that are not based new version of FedML (10 months outdated until today) (2) The evaluation of FedML old version (Oct, 2021) is not based on facts and overlapping with a published paper; (3) unrealistic overlap between system efficiency and data distribution (4) issues on numerical optimization (5) dual submission?

5

Dear Authors of FedScale, I didn't want to comment too much on FedScale because I thought all the experts in the field knew the truth. But you promote your outdated...

chaoyanghe

### Description "Motley: Benchmarking Heterogeneity and Personalization in Federated Learning" (https://arxiv.org/abs/2206.09262) introduces several datasets ### Use case Support Motley in FedScale.

mosharaf

Fix async

1

## Why are these changes needed? 1. Model testing is somehow missing; 2. Weird model accuracy over training; ## Related issue number ## Checks - [ ] I've included any...

fanlai0990

Need a better structure for eval config files (yml), especially job_conf

1

It's completely flat structure under `job_conf` which makes it hard to follow and organize. A better hierarchical organization under this will allow us, for example, to separate aggregator-related params from...

mosharaf

### Description A documentation page that walks through all FedScale parameters, what they mean, and gives some guidance to avoid issues like #160 for example.

mosharaf

### Description Throughout the code, FedScale pickles data before shipping them, which assumes python on both ends. Ideally everywhere, but at the very least, between components, we should avoid pickle...

mosharaf

### Description Currently, FedScale assumes that the device traces have already been pickled first. We should be able to read traces in other formats such as JSON. ### Use case...

mosharaf

Potential dataset

2

https://aml.engr.tamu.edu/book-dswe/dswe-datasets/ @dywsjtu can you look it up? Most of them are really small, but the `4. Wind Spatio-Temporal Dataset2` can be useful with 200 clients. We can discuss here.

mosharaf

Tensorflow data loader example

2

The [tensorflow client example ](https://github.com/SymbioticLab/FedScale/blob/master/examples/tensorflow_engine/tf_client.py) is very helpful, but one suggestion for improvement is to add a [tf Dataset](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) based dataloader. I think the existing dataloaders are all using torch,...

ewenw

1. Clean up aggregator and execution code (@dywsjtu ); 2. Decouple the current aggregator into different modules for a better extension (@fanlai0990 );

fanlai0990