disco
disco copied to clipboard
FedProx implementation
Closes #802
@tomasoignons I rebased the branch onto the latest develop commit. Here is a breakdown of the current commits:
- tfjs allows training via multiple functions: fit, fitDataset, trainOnBatch. We were using fitDataset on each batch so I switched to using trainOnBatch to be consistent and simplify things
- We used to compute logits twice for each update, once for the loss and once for the accuracy. This is very inefficient but not super practical to fix. I implemented one but feel free to suggest a better refactoring
- A temporary commit, I override the training loss by implementing it by hand to then implement the fedprox loss. This commit saves the codebase state where the custom training loss behaves the same as the default loss (make sure this is the case)
- Fedprox loss building on the previous commit, I didn't have the time to test it in depth so this is up to you
As we were discussing, the fedprox sketch has been hard-coded and doesn't allow users to choose which algorithm they want between fedavg and fedprox, and potential future algorithms like Mime.
added a few commits of cleanup:
- cli was using an old version of
immutable(4 vs 5), removing it to use the global one - there was some cli local "node_modules" which shouldn't exist I suspect that it happened during a merge commit
it is still failing but rebasing/merging upon #903 (or "develop" when merged) should allow it to pass.