feature/Updated embedded devices APT command and Docker platform

[sisco0]

The current Pull Request fixes the scenario shown in https://github.com/adap/flower/discussions/975 after installing Docker ARM64 emulation by using QEMU running the commands shown below:

sudo apt-get install qemu binfmt-support qemu-user-static # Install the qemu packages
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes # This step will execute the registering scripts

It has been tested locally under a native x86-64 Linux platform without inconvenience but the run_pi.sh should be tested under a native Raspberry Pi and Nvidia Jetson ARM64 platform as well for verifying that the modified scripts work as expected.

Now we are able to simulate Raspberry Pi and Nvidia Jetson under our environment, which would lead to a better experimentation capability for this example proposal.

A screenshot of the solution working under my x86-64 native environment is attached below.

View proof of evidence

[sisco0]

We need to ensure that the changes do not affect the native client execution for Raspberry Pi and Jetson by testing under those environments, could you please check it up @jafermarq?

[nikita-kotsehub]

I was testing this on RPi 4. I still get the same error as in #975 (where I was also working on RPi 4):

=> ERROR [ 2/11] RUN apt-get update && apt-get install wget -y            0.9s
------                                                                          
 > [ 2/11] RUN apt-get update && apt-get install wget -y:                       
#6 0.780 standard_init_linux.go:228: exec user process caused: exec format error
------
executor failed running [/bin/sh -c apt-get update && apt-get install wget -y]: exit code: 1
Unable to find image 'flower_client:latest' locally
docker: Error response from daemon: pull access denied for flower_client, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.

I did not test the QEMU emulation on a x86-64 Linux platform.

[sisco0]

It seems that the Docker image is not based on your current platform. Could you gently show a docker version output so we could know about the OS/Arch of your server? If we could not go on with this we could create a new Docker image for these requirements basing ourselves on another Docker image at the top of our Dockerfile.

[nikita-kotsehub]

@sisco0, here is the output from running docker version on my RPi 4:

Client: Docker Engine - Community
 Version:           20.10.12
 API version:       1.41
 Go version:        go1.16.12
 Git commit:        e91ed57
 Built:             Mon Dec 13 11:45:14 2021
 OS/Arch:           linux/arm
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.12
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.12
  Git commit:       459d0df
  Built:            Mon Dec 13 11:43:32 2021
  OS/Arch:          linux/arm
  Experimental:     false
 containerd:
  Version:          1.4.12
  GitCommit:        7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[sisco0]

As you might have noticed as well you are currently using a 32 bits version of the Raspberry PI operative system (Raspbian). Is there any chance for you at trying to install the 64 bits version of the operative system? As the base image is intended to run under the 64 bit architecture.

In the case that there is no chance we should create a new Docker image based on 32 bits.

[sisco0]

Confirmation from @nikita-kotsehub or @jafermarq is required for this PR to be merged.

[jafermarq]

Hi @nikita-kotsehub and @sisco0. Thanks for looking into this. I agree adding support for Debian is important since it remains the recommended OS for RPi. I've created a new PR adding it (#1006) and tested it.

@sisco0 , I like the example you provide to simulate the embedded platforms on a standard x86 machine. I tested it and worked fine. Your changes to the bash scripts didn't break anything when running on the devices :) .

Should we instead clearly state in the code snippet you add to the README that using the emulation is fine for a quick test but is not recommended for running things. I've tested this myself an it runs considerably slower (even on a somewhat powerful CPU -- See attached). Also, it might be worth labelling this as "advanced"? or maybe move it to a new directory inside the example? What do you think?

[nikita-kotsehub]

@jafermarq thank you for looking into this and making the updates!

I am still getting the same error from #975 when running ./run_pi_debian.sh --server_address=<server_ip> --cid=0 --model=Net:

executor failed running [/bin/sh -c apt-get install wget -y]: exit code: 100
Unable to find image 'flower_client:latest' locally
docker: Error response from daemon: pull access denied for flower_client, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.

I am not sure if the issue is on my side (do I need to pull the image manually from somewhere?) or with some of the repo code. Could you please take a look at it and let me know if there is anything I shall do on my side? Thanks!

[sisco0]

@nikita-kotsehub comments on the new PR would fit better in it, just for being able to reference this in code review comments. Kindly take care of the following parameter in the screenshot below. As you could wisely discover, the name of the image built is different from flower_client. My advice would be to make both of them meet by taking the arrow referenced one as base.

Could you kindly test if changing the image name in the run_pi_debian.sh leads to a successful implementation by your side?

[jafermarq]

Closing this since the embedded_devices example was updated recently, adding quite a few simplifications and making it more versatile overall. Please feel free to open a new issue if you encounter some problems!