microkit icon indicating copy to clipboard operation
microkit copied to clipboard

Add support for domain scheduling

Open JE-Archer opened this issue 1 year ago • 2 comments

This PR aims to add support to the Microkit for seL4's domain scheduling.

Ideally, if the user wished to use domain scheduling, they would specify a domain schedule as well as a mapping of PDs to domains in their system description. However, implementing this is currently not possible as domain scheduling in seL4 requires schedule configuration to be done at kernel build time, which conflicts with the SDK model of Microkit where SDKs are shipped with prebuilt kernel images. More specifically, three kinds of information must be provided to the kernel in order to use domains:

  • the number of domains;
  • the timeslice of each domain;
  • the domain of each TCB.

The last can be provided to the kernel at runtime using the domain cap, but the former two are provided at build time. We assume that in the future the seL4 kernel will allow full runtime specification of all of the above, in which case there will be no issue. Until then however, if we want people to be able to use domains with the Microkit, we need something a bit hackier.

The solution this PR implements is based on patching the values of the ksDomSchedule and ksDomScheduleLength variables in the seL4 kernel ELF. The SDK will patch these variables in accordance with a user-provided domain schedule and PD-domain mapping. This PR also adds support to the SDF for the user to provide this information. The SDF syntax looks like:

<system>
  <domain_schedule>
    <domain name="domain0" length="100" />
    <domain name="domain1" length="200" />
  </domain_schedule>

  <protection_domain name="pd0" priority="1" domain="domain0">
    <program_image path="pd.elf" />
  </protection_domain>

  <protection_domain name="pd1" priority="1" domain="domain1">
    <program_image path="pd.elf" />
  </protection_domain>
</system>

The domain_schedule element allows enumerating a sequence of domains along with each domain's timeslice. The order of the sequence is the order of scheduling. The protection_domain element now has an additional attribute, domain, which is a string which must match the name of one of the domains in the domain schedule. In seL4, domains are identified by integers, but here we allow users to represent them as strings, and map them to integers internally.

This PR also adds the --experimental-domain-support flag to the build_sdk.py script, to enable the above support. This flag will also cause the script to build each seL4 kernel image with the relevant domain-related kernel build flags. In order for this PR to work, the kernel must be patched to remove the const qualifiers from the ksDomSchedule and ksDomScheduleLength variables.

JE-Archer avatar Jul 17 '24 04:07 JE-Archer

Another thing is that if two PDs are in separate domains, I believe they cannot communicate in anyway, so we would also need to validate there are no channels between them.

In terms of functionality they can communicate freely with all seL4 mechanisms. Restrictions only come into play if you want to use the information flow theorems and at the same time achieve one-way information flow (as opposed to two-way) between domains. For one-way flow, only shared memory (with read-only on one end) and notifications are allowed. If you want to prevent all information flows, then they of course can't communicate at all.

lsf37 avatar Jul 18 '24 06:07 lsf37

In terms of functionality they can communicate freely with all seL4 mechanisms. Restrictions only come into play if you want to use the information flow theorems and at the same time achieve one-way information flow (as opposed to two-way) between domains. For one-way flow, only shared memory (with read-only on one end) and notifications are allowed. If you want to prevent all information flows, then they of course can't communicate at all.

Ah okay, my bad then.

Ivan-Velickovic avatar Jul 18 '24 06:07 Ivan-Velickovic