Create separate containers for testing Open AD Kit

[xmfcx]

Checklist

• [X] I've read the contribution guidelines.
• [X] I've searched other issues and no duplicate issues were found.
• [X] I've agreed with the maintainers that I can plan this task.

Description

I will create 2 docker containers that will pull the universe and build selected packages separated by their basic functionality.

Container contents:

• Container 1:
  • drivers, sensing, perception, localization
• Container 2:
  • planning, control

The reason we split them this way is to ensure we minimize the data transfer overhead between containers.

Purpose

To prototype the first multi-container version of the Autoware Universe for Open AD Kit.

Possible approaches

I will first test them out on my local machine. Then maybe we can set-up the CI but I am not sure yet.

Definition of done

• [ ] Decide what packages are required for the container

• [ ] First container:

  • [ ] Create the dockerfile like in here: https://github.com/autowarefoundation/autoware/blob/main/docker/autoware-universe/Dockerfile
  • [ ] Make sure the respective tutorials run with the autoware builds:
    • https://autowarefoundation.github.io/autoware-documentation/main/tutorials/ad-hoc-simulation/rosbag-replay-simulation/

• [ ] Second container:

  • [ ] Create the dockerfile like in here: https://github.com/autowarefoundation/autoware/blob/main/docker/autoware-universe/Dockerfile
  • [ ] Make sure the respective tutorials run with the autoware builds:
    • https://autowarefoundation.github.io/autoware-documentation/main/tutorials/ad-hoc-simulation/planning-simulation/

[xmfcx]

Created branch https://github.com/autowarefoundation/autoware/tree/openadkit-containers to work on this, will create a PR when I start making changes.

[ghost]

The original issues created for this work are below:

• https://github.com/autowarefoundation/autoware.ai/issues/2447
• https://github.com/autowarefoundation/autoware-documentation/issues/101
• https://github.com/autowarefoundation/autoware-projects/issues/36
• https://github.com/autowarefoundation/autoware-projects/issues/37

The top two issues relate to creating some kind of automated build process for Autoware as a microservices architecture, and then the bottom two issues are for the initial microservices PoC work, that assumed that we would be splitting Perception into its own container.

[ghost]

@xmfcx Once the containers have been created, I am happy to help out with the testing part of this issue.

[xmfcx]

Required nodes

In order to list the required nodes for the containers, I am trying to use @kenji-miyake -san's tool from https://github.com/orgs/autowarefoundation/discussions/2581

The tool is private for now but I'm working in cooperation with him to resolve some issues on the package.

With the tool, I will hopefully have a list of all the nodes that are launched with the main launch files.

Building the docker image manually

Right now I am trying to use this build.sh to build the docker image but every time I tried, I got some connectivity issues and every time I restarted, it failed at some point and takes a very long time to retry. I will investigate this further.

I will try to build the container without buildx thingy, maybe add more layers to the image, also will consult to @kenji-miyake on this.

[ghost]

@xmfcx Hi Fatih. Would it be possible to update this issue with your current progress please? 😄

[ghost]

@xmfcx Hi Fatih. Based on the conversation last week, it seems as if this issue is no longer considered to beneficial towards developing a microservices architecture for Autoware and the CI/CD infrastructure changes that would need to be made to support that architecture. If so, could you please add a short summary note and close the issue please.

[xmfcx]

Hello @LalithVipulananthan,

Yeah, like you've said, this issue wouldn't achieve much. What we really need is to have a solid package release process based on bloom.

[doganulus]

I wonder what prevents to build containers as specified in this discussion.

Honestly speaking, it is hard to argue that ROS tooling is modern by current standards. I am afraid of relying more on that would bring more headache. Containers seemed like a good solution to me. What went wrong exactly?

[xmfcx]

@doganulus Autoware Universe currently provides several docker containers in https://github.com/autowarefoundation/autoware/pkgs/container/autoware-universe and you can follow the instructions from the documentation to run Autoware using them.

These containers are built as artifacts of the Autoware CI workflows, twice in a month.

Also the plan is create our own debian repository for Autoware ROS packages' releases. This way if we want to create containers from select few packages, we can install them into the container using apt. Any other ROS package is released/installed that way, including many dependencies of the Autoware.