Supporting config.json generation for non-Linux OCI images

[mateuszkwiatkowski]

Hello,

When creating image on FreeBSD I have to temporarily mark image as Linux to workaround this error:

# umoci unpack --image freebsd:13.0-BETA4 bundle-freebsd-13.0-BETA4
   ⨯ create runtime bundle: unpack config.json: generate config.json: unsupported OS: freebsd
# umoci config --image freebsd:13.0-BETA4 --os linux
# umoci unpack --image freebsd:13.0-BETA4 bundle-freebsd-13.0-BETA4
# echo "it worked!"

I think this error message can be disabled. The whole procedure of creating image on FreeBSD is documented here: https://gitlab.com/runhyve/jailer/-/snippets/2089767

[cyphar]

The issue is that umoci unpack has to create a config.json (an OCI runtime-spec configuration file) which means that we have to generate said file for whatever operating system the container image uses (because the assumption is that you will usually want to runc run -b bundle ctr immediately after umoci unpack).

I will need to look into how the OCI runtime-spec has been used by FreeBSD containers to figure out how the config.json should look on FreeBSD. If you just need to extract the image and don't care about running it as a container you can do umoci raw unpack which will unpack an image to a directory and not do config.json generation -- that should work with FreeBSD images AFAIK.

[mateuszkwiatkowski]

Hi @cyphar, Thank you for response. There's no OCI runtime on FreeBSD yet. I'm running these images with a jail(8) utility which only needs rootfs from the image. In the future there would be a wrapper that could use OCI metadata and pass it as a configuration and environment to jail(8).

[mateuszkwiatkowski]

Hi @cyphar,

I have just learned about this project that provides runtime for FreeBSD: https://github.com/samuelkarp/runj Unfortunately raw unpack is not a solution because config.json is required to configure working container. Here is a snippet with workaround: https://gitlab.com/runhyve/jailer/-/snippets/2092326

[cyphar]

Okay, I'll cook something up then.

[samuelkarp]

@cyphar I'm happy to help here if you want any.

Edit: I'm currently using a significantly trimmed-down config in runj, with any unknown fields (most of them!) ignored. The example config (generated by runj demo spec, similar to runc spec) is similarly basic.

[mateuszkwiatkowski]

To move this forward for me I created trimmed-down config for FreeBSD: https://github.com/runhyve/umoci/commit/759dccf4cbbe47a54ca2ef3d44d9c3eb83d7a157 I'm not sure if this is something you might want to merge into umoci. What do you think about creating a #freebsd channel in OCI Slack?

[mateuszkwiatkowski]

Hi @cyphar,

There's now a #freebsd channel in OCI's Slack.

[ZoltanW]

I was hoping to use umoci for managing (unpack specifically) OCI images created with buildah. The images are created for an RTOS, not Linux. I ran into the issues reported here, as well as other issues related to mounts, namespaces, etc - all strongly tied to Linux images. Any plans for improving the platform/OS abstraction in umoci to better support non-Linux OCI images?

[cyphar]

I would be happy to do it, though I would need some guidance as to what the right conversions are for images to runtime-specs for each individual platform. My guess is that most of them can be copied as they are normally, but we'd need to have a good source of defaults for non-Linux platforms (the current defaults for Linux are based on runc which only supports Linux).

Do you know of a good source of runtime-spec defaults for non-Linux platforms? (One issue is that a lot of runc implementations for other platforms re-use the "linux" namespace when they're not running on Linux, so it's unclear what is the best thing to do for such platforms.)

[ZoltanW]

Just for the record, extraction of the rootfs works fine. The problem is only with the generation of the config.json. Let me start with attaching a typical config.json file used on the RTOS.

config.json.txt

A few notes: the process is always an application, not the shell (in the example, it's a python application). The mounts are always specified explicitly by the image (there are no defaults), and are typically "iodev" type, such as the one in the example. The format is: VOLUME "/dev/urandom:/dev/urandom:--:iodev" which result in:

    "mounts":[{
        "destination":"/dev/urandom",
        "source":"/dev/urandom",
        "type":"iodev",
        "options":["--"]
    }]

There can be a few optional parameters specified; for example, in the Dockerfile:

   LABEL com.windriver.vxworks.rtp.rtpStackSize 0x40000
   LABEL com.windriver.vxworks.rtp.rtpPriority  50

which results in the config.json:

    "vxworks":{
        "rtp":{
            "rtpPriority":"50",
            "rtpStackSize":"0x40000"
        }
    }

We can certainly discuss the details later, this is just an intro to give an idea of the scope.

[ZoltanW]

By the way, please let me know if you prefer that I open a new issue. I just started commenting here because it is a similar topic - support for non-Linux platform.

[cyphar]

Since you have an example config, I guess there are programs which generate these configs already? If so we can probably just use their defaults (assuming they're Apache-2.0-compatibly licensed ofc) -- how did you generate the config? And yeah it's perfectly okay to comment here, non-Linux configs is sort of the point of this issue (I'll update the title).

[ZoltanW]

Since you have an example config, I guess there are programs which generate these configs already?

There is unpack capability implemented in C, building on the RTOS APIs (and proprietary commercial license). There is also not much in terms of default, it is all created by interpreting the image content. So I don't really think there is much that could be reused for umoci.

While experimenting with similar changes that Mateusz made for BSD, I came up with something like this:

func Example_VxWorks() rspec.Spec {
	return rspec.Spec{
		Version: rspec.Version,
		Root: &rspec.Root{
			Path:     "rootfs",
			Readonly: false,
		},
		Process: &rspec.Process{
			Terminal: true,
			User:     rspec.User{},
		},
		VxWorks: &rspec.VxWorks{
			Resources: &rspec.VxWorksResources{
                        },
                },
	}
}

I also had to disable Linux specific code from UnpackRuntimeJSON(). And finally I modified MutateRuntimeSpec() to parse volume info from image instead of hard-coded values:

	for vol := range ig.ConfigVolumes() {
                parts := strings.Split(vol, ":")
                dest, source, typ := parts[0], parts[1], parts[3]
                opts := []string{parts[2]}
		spec.Mounts = append(spec.Mounts, rspec.Mount{
			Destination: dest,
			Type:        typ,
			Source:      source,
			Options:     opts,
		})

I have not found a way to parse and generate the proprietary LABEL values (see rtpPriority and rtpStackSize) above.

[ZoltanW]

Is there any expected time frame this issue may get addressed? Or perhaps info on prioritization relative to other open issues? TIA.

[cyphar]

I still need some kind of spec or other documentation explaining what the relevant fields and labels are so we can convert them. By the way, rspec.VxWorks doesn't exist in the upstream runtime-spec repository so I'm not sure what that is supposed to be either? I guess if the rule is just strip "com.windriver." and then that is the JQ-like path in the JSON object, I guess we could implement that fairly easily.

I also am slightly concerned about the VOLUME conversion -- that particular usage of VOLUME is outside of the scope of the image-spec (not to mention it increases the security risk of VOLUME above and beyond what it is normally -- being able to specify mount arguments directly from an image is placing a lot of trust in your images). I guess if it was only enabled for this one weird platform that might be okay, but it still gives me cause for concern...

All-in-all, without having access to this RTOS system (or at least some images from it), I'm not entirely sure how many things need to be changed. But I can take a look at this again next week.