Tensorflow data loader example

[ewenw]

The tensorflow client example is very helpful, but one suggestion for improvement is to add a tf Dataset based dataloader. I think the existing dataloaders are all using torch, and this client converts torch tensors into tf tensors, which is less efficient.

[fanlai0990]

Thanks for raising it up. Yes, the current dataloader is less efficient in my test, and I agree we'd better support naive tf dataloaders.

However, it may take a while to do so for all datasets.

• Do you have some preference regarding which datasets/tasks to prioritize? One workaround, for now, is to prefetch the next batch in order to hide this latency;
• Wondering whether tensorflow (or JAX) is dominant in your use? While FedScale can support tf, optimizing it is not our recent top-priority focus (we are actively optimizing for the scalability and various modules like communication layers), but we very welcome the contribution and are happy to help.

[ewenw]

There is no preference on which dataset since we will be building a custom data loader for our internal dataset. I just think it would be useful to have an example for reference in examples/tensorflow_engine.

And yes, tensorflow will be dominant in our use case, and we are happy to contribute as we make progress!