rFaaS
rFaaS copied to clipboard
spcl
Add Docker registry
We are wrapping up the support for using Docker images and rFaaS (#10). However, we only use DockerHub, which has multiple restrictions - throttling and network latency. We want to locally deploy a Docker registry.
- [ ] Add scripts for deploying the Docker registry and generating JSON configuration.
- [ ] Add configuration of Docker registry in the rFaaS executor manager configuration.
- [ ] Add polling Docker images on container allocation in the execution manager - only when the image is not present already.
- [ ] Support registry credentials.
@BhuvanAde Absolutely, please feel free to start on the issue :-)
Let me know if you have any questions or concerns - happy to help.
Hi @mcopik. I have made a great deal of progress on this issue so far (as well as some progress on #10). However, I am wondering what the docker network configuration used during testing looks like. I see that in the hard-coded part of the code where the docker_rdma_sriov binary is invoked (here), the docker network mynet is specified. I presume that this network is a user-defined bridge network that was used during testing/evaluation. Do you remember how this network was defined?
UPDATE: docker network create -d sriov --subnet=<subnet> -o netdevice=<device> mynet seemed to work.
@mattnappo Great to hear that - thanks so much for the help! Can you please open a draft PR? I was going to work on the same thing (fixing container selection and customizing the hardcoded options), and I do not want to repeat your work.
Yes, you found the correct thing - this is the command. I'm afraid that our documentation does not contain references to the installation instructions for the Mellanox plugin - I need to fix that.
In addition to using the SR-IOV plugin, which is unavailable on many platforms and can be a bit fragile, we also used standard Docker with host networking - I will add it as another customization.
@mcopik Here is the draft pr. I plan on finishing up the PR within the next few days. Also: I was able to setup a rFaas cluster on an AWS VPC and perform tests. I built a custom kernel to use Soft-RoCE since my cluster did not have RDMA-capable NICs. If you were wondering if rFaas works over Soft-RoCE, it does!
@mattnappo Thanks, glad to hear it works! :-) I will review the PR soon.
Did you manage to run SR-IOV on top of SoftRoCe? Or did you try Docker host networking mode?
Yes, I ran SR-IOV on SoftRoCe. I can try to get it working with host networking.