Add `CollaboratorSerialiser` middleware for serialisation/deserialisation between collaborator and aggregator client
Summary
Moving serialisation/deserialisation logic out of collaborator into a separate middleware.
Why?
By clearly separating concerns across layers, it aims to enhance code readability for contributors and improves debuggability in case of failures. This middleware introduction is similar to "middleware chaining" paradigm followed widely in Go where the same function is implemented across multiple "layers", each handling a distinct aspect of the functionality. In the context of this PR, this is done such that
- Layer 1 (
Collaborator): Performs operations central to FL. - Layer 2 (
CollaboratorSerialiser): Performs serialisation/desrialisation of numpy array. - Layer 3 (Transport): Interacts with the aggregator server to send/receive data.
Type of Change (Mandatory)
- Feature enhancement
- Restructuring
Description (Mandatory)
- The collaborator uses
TensorCodecto convert tensors to protobuf format and vice versa in order to communicate with the aggregator. - This conversion logic is moved out from the collaborator and it focuses solely on the core functionality while the conversion and client call is abstracted from the user.
- With the introduction of other transport implementations in the future, the functionality can be retained by passing the new (say REST) client to the middleware.
Testing
Manual and CI. Eden compression CI
Additional Information
- Will be adding a serialisation middleware to the aggregator in a subsequent PR.
The changes in tests mostly include moving tests between the now appropriate modules and/or renaming them.
To begin with, could you add a "why" section in the PR description? In addition to the obvious encapsulation benefits, what framework capabilities do you expect this refactoring to enable down the road?
@teoparvanov Thanks for the comments! Updated the PR description. Adding a bit more descriptive answer here. This PR doesn't directly support any upcoming features/capabilities. However, by clearly separating concerns across layers, it enhances code readability for contributors and improves debuggability in case of failures. For instance, future developers won’t need to worry about low-level details like converting NumPy arrays to named tensors, as such operations are abstracted away and don’t impact the core functionality of the component. This is a small but hopefully meaningful step toward the larger improvements we’re aiming for in the repo.