Problem enabling MPTCP on CBL-Mariner 2.0

[ohault]

Linux cbl-mariner 5.15.41.1-1.cm2

sysctl -w net.mptcp.enabled=1 sysctl: cannot stat /proc/sys/net/mptcp/enabled: no such file or directory

[elsaco]

@ohault a quick grep through 5.15.41.1-1.cm2 kernel config shows CONFIG_MPTCP is not set. The kernel needs to be rebuild with Multipath TCP enabled.

[ohault]

1. How to do it ? Setting CONFIG_MPTCP=y from https://github.com/microsoft/CBL-Mariner/blob/2.0/SPECS/kernel/config will work, but how to get the associated tar.gz file for 5.15.41 kernel as it is not published on https://github.com/microsoft/WSL2-Linux-Kernel (ref. https://github.com/microsoft/CBL-MarinerDemo#modify-the-demo-image-kernel )

2. Does is possible to do it from within a running CBL-Mariner instance ?

[elsaco]

1. the 5.15.41 SRPM is on packages.microsoft.com at:

https://packages.microsoft.com/cbl-mariner/2.0/prod/base/srpms/kernel-5.15.41.1-1.cm2.src.rpm

2. yes, you can build the kernel using a CBL-Mariner instance, just need to install the dev tools if missing.

[ohault]

1. I'm making progress but I'm currently blocked with errors, I have just created this Issue
2. It's not clear for me, if I have to recompile the kernel from https://github.com/microsoft/CBL-Mariner-Linux-Kernel/releases/tag/rolling-lts%2Fmariner-2%2F5.15.41.1 and how to switch to the new one in my running CBL-Mariner 2.0 instance ?

[elsaco]

I pointed to that SRPM because it is the source of the kernel in CBL-Mariner 2.0. After you add the SRPM please take a look in /usr/src/mariner and in the SOURCES folder there's kernel-5.15.41.1.tar.gz:

tux@mariner-vm1 [ ~ ]$ sudo tree -L 2 /usr/src/mariner
/usr/src/mariner
|-- SOURCES
|   |-- cbl-mariner-ca-20211013.pem
|   |-- config
|   |-- config_aarch64
|   |-- kernel-5.15.41.1.tar.gz
|   `-- sha512hmac-openssl.sh
`-- SPECS
    `-- kernel.spec

Inside the SPECS folder is the kernel.spec file that is used to build the new RPM. The file contains also instructions how to add your own config changes:

# When updating the config files it is important to sanitize them.
# Steps for updating a config file:
#  1. Extract the linux sources into a folder
#  2. Add the current config file to the folder
#  3. Run `make menuconfig` to edit the file (Manually editing is not recommended)
#  4. Save the config file
#  5. Copy the config file back into the kernel spec folder
#  6. Revert any undesired changes (GCC related changes, etc)
#  8. Build the kernel package
#  9. Apply the changes listed in the log file (if any) to the config file
#  10. Verify the rest of the config file looks ok
# If there are significant changes to the config file, disable the config check and build the
# kernel rpm. The final config file is included in /boot in the rpm.

Make the necessary changes and copy config into /usr/src/mariner/SOURCES and use rpmbuild -tb for binary package only or rpmbuild -ta for everything. After a succesful build install the new kernel RPM and reboot.

[ohault]

Thank you @elsaco. By writing 'After you add the SRPM' , do you mean manually using wget, rpm2cpio and cpio, or is there on purpose script to use from the Mariner toolchain ?

To compile the Kernel under Mariner 2.0, I cannot find libssl-dev and libelf-dev packages. Should I use others packages or add another source ?

[elsaco]

I'm using a CBL-Mariner vm built with the ISO because it has all the tools installed already, i.e. compiler and more. The same can be done with a container. After pulling kernel-5.15.41-1.cm2.src.rpm with wget or curl install it with rpm -i. All the files will added to /usr/src/mariner.

On first try it failed to build kernel-tools due to unmet dependencies not specified in kernel.spec:

Processing files: kernel-tools-5.15.41.1-1.x86_64
error: File not found: /usr/src/mariner/BUILDROOT/kernel-5.15.41.1-1.x86_64/usr/lib64/libperf-jvmti.so

Because only MPTCP was enabled the simple way is to edit /usr/src/mariner/SOURCES/config and around line 1127 remove # CONFIG_MPTCP is not set and add:

CONFIG_MPTCP=y
CONFIG_MPTCP_IPV6=y

If more changes are to be made use make menuconfig because there might be dependencies which are easy to miss when editing manually.

Build the new kernel with rpmbuild -bb SPECS/kernel.spec and the packages will be in:

/usr/src/mariner/RPMS/x86_64/kernel-5.15.41.1-2.x86_64.rpm
/usr/src/mariner/RPMS/x86_64/kernel-tools-5.15.41.1-2.x86_64.rpm
/usr/src/mariner/RPMS/x86_64/kernel-drivers-sound-5.15.41.1-2.x86_64.rpm
/usr/src/mariner/RPMS/x86_64/kernel-devel-5.15.41.1-2.x86_64.rpm
/usr/src/mariner/RPMS/x86_64/bpftool-5.15.41.1-2.x86_64.rpm
/usr/src/mariner/RPMS/x86_64/kernel-drivers-accessibility-5.15.41.1-2.x86_64.rpm
/usr/src/mariner/RPMS/x86_64/python3-perf-5.15.41.1-2.x86_64.rpm
/usr/src/mariner/RPMS/x86_64/kernel-docs-5.15.41.1-2.x86_64.rpm
/usr/src/mariner/RPMS/x86_64/kernel-debuginfo-5.15.41.1-2.x86_64.rpm

Inside kernel.spec I changed the Release to 2 just to differentiate from the default kernel package. Install kernel-5.15.41.1-2.x86_64.rpm and reboot.

MPTCP is now enabled:

root@mariner-vm2 [ /usr/src/mariner ]# sysctl net.mptcp.enabled
net.mptcp.enabled = 1

List of packages needed to build kernel-tools:

systemtap-sdt-devel
libunwind-devel
libcap-devel
libnuma-devel
libbabeltrace2-devel
msopenjdk-11

[ohault]

Thank you again @elsaco , It has been a long story, because I was using the core package installation and my VM was configured to use secure boot with Microsoft UEFI Certificate Authority (bad shim signature error message once rebooted).

Once all the missing packages installed, I have been able to build a new kernel version with MPTCP.

[ohault]

To streamline this process, I have just created the following ticket https://github.com/microsoft/CBL-Mariner/issues/3550

[Malateshk007]

@ohault, We are interested in confirming whether you are still considering the activation of the MPTCP feature in the kernel. With great anticipation, we would like to inform you that a major new release is imminent. In light of this, would it be agreeable to you if we focus our configuration assessment on the forthcoming release rather than the existing stable version?

[suhuruli]

@christopherco, @Malateshk007, @eric-desrochers - we spoke with @ohault in our community call and we agreed to take a hard look at enabling this config by default for perf benefits in 3.0. This would be in-line with almost all popular Linux distributions.

Let's track this in the feature dashboard!

[eric-desrochers]

[2.0] Note that we won't introduce such change during 2.0 lifetime to maintain stability.

[3.0] But it's a perfect timing to discuss introducing it in Azure Linux 3.0 as we are still in devel phase. I'll discuss with the Azure Linux kernel dev and will get back to you on this.

Additionally, this seems supported/enabled in recent major distros nowadays. Reference: https://www.mptcp.dev/

Thanks !

[eric-desrochers]

@christopherco for vis.