FedScale icon indicating copy to clipboard operation
FedScale copied to clipboard

Dataloader support for TF

Open kashprime opened this issue 1 year ago • 4 comments

Description

Hi Team, there is admittedly some overlap between this issue and a previous one. However I thought I would make a new one since the other is fairly old. I am looking for some data loader support in FedScale for TensorFlow. It seems the data classes as they stand are written using PyTorch in mind and I was wondering if anyone has any experience using a TensorFlow data set, particularly using .tfrecord files with a known schema.

Use case

I work in industry and am looking to use FedScale to run a simulation of a specific federated learning model on a specific set of hardware. All of our models are written using tf and keras.

kashprime avatar Dec 05 '22 20:12 kashprime

Hi kashprime, we had some tf data loader use cases (not public yet), we will discuss about it by this week and you may hear back soon.

AmberLJC avatar Dec 05 '22 20:12 AmberLJC

Awesome, thank you AmberLJC!

kashprime avatar Dec 05 '22 20:12 kashprime

Hi kashprime, Thank you for trying FedScale. You can refer here for a quick workaround too, which transforms PyTorch to TF tensors.

fanlai0990 avatar Dec 07 '22 01:12 fanlai0990

Hi @kashprime. We have some updates to share with you: we are actively collaborating with LinkedIn to add TF support. We will merge this #194 soon to support TF.

fanlai0990 avatar Jan 13 '23 23:01 fanlai0990