Graal Native Image Support

[brett-smith]

This issue is to discuss possible support for Graal Native Image, and how that might be achieved.

Of late, I have been using Graal Native Image more and more, often along with dbus-java, particularly on Linux. For example, if you add PicoCLI it makes Java a great language for writing DBus based command line utilities that are easy to distribute, fast to start up and use a lot less memory. You can even totally statically link (e.g. with libc or musl). Who'd have thought it.

The main challenge to using native image, is providing the reflection (and other) meta-data to help Graal in it's tasks of examining every possible code path. This has always been a bit painful, but has improved of late with things such as the meta-data repository and better tools to automate this. Lots of libraries now either include Graal meta-data, or there are rebuilds of libraries that have had meta-data added (Quarkus etc).

Getting good Graal support in dbus-java itself is going to involved a few different tasks. Its mainly about some resource files, and a new tool.

Checking Dependencies

To have good support, all 3rd party dependencies will need to be checked if they require meta-data, and if they provide it. Either themselves or via an external means. I know JNA has it, other dependencies (in transports I think primarily) will need to be checked. The pure Java (17) one I don't think requires any additional meta-data.

Library Meta-data

Then there is the library itself. Adding meta-data to the project is done by adding .json files to src/main/resources/META-INF/native-image/[xxxxxx]/yyyyy.json, where xxxxxx is a project name (e.g. dbus-java) and yyyyyy is the one of the classes of meta-data.

You can use the native image tracing agent, to generate the meta-data at run-time, but this can generate a lot of platform specific cruft that often is not required. It is very helpful though, and we've found fine in production if a brute-force approach is acceptable.

The classes of meta-data are ..

• jni-config.json - contains data about JNI methods called and libraries used
• predefine-classes-config.json - not sure about this one, i've not yet seen it populated
• proxy-config.json - data about Proxy implementations (dbus-java uses this one)
• resource-config.json - data about classpath resources loaded. There are sometimes surprising things loaded as resources.
• relect-config.json - data about things that need to be reflectable (dbus-java uses this one)
• serialization-config.json - data about Serializable usages

So far, I've only discovered two types of meta-data that are needed for the basic library. proxy-config.json and reflect-config.json. That is not to say there are not others, but this is all i've needed so far.

proxy-config.json

[
	{
		"interfaces": [
			"org.freedesktop.dbus.interfaces.DBus"
		]
	},
	{
		"interfaces": [
			"org.freedesktop.dbus.interfaces.Properties"
		]
	},
    {
        "interfaces": [
            "org.freedesktop.dbus.interfaces.Introspectable"
        ]
    }
]

reflect-config.json

[
{
  "name":"org.freedesktop.dbus.interfaces.DBus$NameAcquired",
  "queryAllDeclaredConstructors":true,
  "methods":[{"name":"<init>","parameterTypes":["java.lang.String","java.lang.String"] }]
},
{
  "name":"org.freedesktop.dbus.interfaces.Introspectable",
  "allDeclaredClasses":true,
  "queryAllDeclaredMethods":true
},
{
  "name":"org.freedesktop.dbus.interfaces.Peer",
  "allDeclaredClasses":true,
  "queryAllDeclaredMethods":true
}
]

Annotation Processor or Code Generator

Lastly, it would be great if dbus-java also provided the tools for users to generate meta-data for their own DBusInterface implementations, and other DBus structures.

For example, DBusInterface implementations must be both declared in proxy-config.json and must be fully reflectable for dbus-java to function. Struct must also be fully reflectable.

This could be done in InterfaceCodeGenerator, with it generating Graal meta-data along with annotated Java source, but I feel this would be better solved using a separate Annotation Processor, applied at a later time. This means hand-written dbus-java interfaces and structures would also be handled.

This is how PicoCLI solves Graal integration, by looking for it's own annotations and generating the appropriate json resources at build time. Annotation processors are a Java feature, and tools like Maven and others have well documented features for enabling processors.

However, an "Annotation Processor" is exactly that. It finds annotations. In dbus-java, annotations for example on a DBusInterface are optional. For this reason, it may be better to have a new annotation, e.g. @DBusNative that tags either a DBusInterface or Struct as a candidate for generation of Graal meta-data.

Risks

None really. Any changes should be entirely non-invasive, and not require any particular version of Java. There would only be a single new annotation.

Some transport providers may require either additional meta-data.

Graal native image developers should be by now well used to tweaking or providing additional meta-data while everybody catches up.

Conclusion

Graal native image really is awesome. It is going to be a significant part of Javas future, so it would be great to get 1st class support here.

If the idea sounds good to you, I will at some point get a PR together. I have most of the parts to achieve all of the above, it just needs bringing together in dbus-java itself.