[cpp] Marshalling Extern Types

[Aidan63]

Another long one, so make sure you're sitting comfortably.

Corresponding hxcpp PR: https://github.com/HaxeFoundation/hxcpp/pull/1189

The Problem

Working with the current interop types comes with many pitfalls which are not immediately obvious, these include.

• Externed value types (cpp.Struct / cpp::Struct) do not work as expected when captured in a closure. If you mutate one of these captured value types you are mutating a copy, not the object referenced by the variable name.
• These struct types only really work with C structs, not C++ structs and classes which might have all sorts of interesting constructors, destructors, assignment operators, etc. Structs are compared using memcmp and copies are made using memcpy, so if you have a C++ class which requires copy constructors, copy assignment, or others to work correctly, things will go wrong in ways which are painful to debug.
• Leaking objects is very easy as destructors are not consistently called. If you place one of these structs in a class field its destructor will never be called.
• Value types stored in classes will be initialised to zero, not have any default constructor invoked, so any default values or setup code will not be invoked.
• It’s very easy to write code which does different things depending on the debug level and optimisations enabled by the compiler. You can easily find yourself modifying a copy of a struct in a temporary variable the compiler created, creating very hard to track down bugs.
• Common C/C++ patterns, such as pointers to pointers, are hard to extern without lots of boilerplate and glue code wrangling.
• All extern classes are considered native by the generator, so if you want to extern a custom hx::Object sub class some of the necessary GC code isn’t generated for the generational GC.
• Many other things!

In short, the current interop handlers mostly work with basic c structs in local variables, but if you want to interop directly with C++ classes, you’re going to have a painful time in anything but the most basic cases.

If you just want to see a quick example of it all in action here's a gist which will compile on Windows and use DXGI to print out all displays connected to the first graphics adapter. DXGI uses pointers to pointers, pointers to structs, pointers to void pointers, and other C++ concepts which have been very difficult to map onto haxe in the past. But hopefully you'll agree that it looks like pretty "normal" haxe.

https://gist.github.com/Aidan63/07364c227335f02fbe50b9c9576f7544

New Metadata

In my mind there are three categories of things you might want to extern, native value types, native pointer types, and "managed" (custom hx::Object sub classes) types. This merge introduces three new bits of metadata to represent these categories and solve the above issues.

cpp.ValueType

Using the @:cpp.ValueType metadata on an extern class will cause it to be treated as a value type, so copies will be created when passing into functions, assigning to variables, etc, etc.

@:semantics(reference)
@:cpp.ValueType({ type : 'foo', namespace : [ 'bar', 'baz' ] })
extern class Foo {
	var number : Int;

	function new() : Void;
}

function main() {
	final obj  = new Foo();
	final copy = obj; // creates a copy
}

I've chosen the metadata to take a struct which currently supports a type field for the underlying native type name and namespace which must be an array of string literals for the namespace the type is in. If type is omitted then the name of the extern class is used, if namespace is omitted then the type is assumed to be in the global namespace.

Using this metadata provides several guarantees old struct types. First it behaves how you'd expect when captured in a closure.

function main() {
	final obj  = new Foo();
	final func = () -> {
		obj.number = 7;
	}

	func();

	trace(obj.number); // 7 will be printed.
}

Destructors are guaranteed to be called. When a value type is captured in a closure, stored in a class field, enums, anons, or any other GC objects it is "promoted" to the GC heap and the holder class its contained within registers a finaliser which will call the destructor.

Operators on the defined native type are always used, no memcmp or memcpy. Copy constructors, copy assignment, and standard equality operators are always used no matter the case.

The same sort of null checks are performed with references to these value types as standard haxe classes so you will get standard null exceptions instead of the program crashing with a memory error.

The nullability of these values types is unfortunately a bit odd... If you have a explicitly nullable TVar value type then its always promoted and can be null. But a null value type doesn't make much sense so I've disallowed value type variable declarations with no expression or with a null constant. Trying to assign a null value to a stack value type will result in a runtime null exception. Since value types in class fields and the likes are always promoted they are null if uninitialised. Ideally value type externs could have the same "not null" handling as core type abstracts, but that doesn't seem possible.

Interop with the existing pointer types is also provided as well implicit conversions to pointers on the underlying types for easier interop.

This value type metadata is also supported on extern enum abstracts. Historically externing enums have been a bit of a pain but it works pretty nicely now.

@:semantics(reference)
@:include('colour.hpp')
@:cpp.ValueType({ type : 'colour' namespace : [ 'foo' ] })
private extern enum abstract Colour(Int) {
    @:native('red')
    var Red;

    @:native('green')
    var Green;

    @:native('blue')
    var Blue;
}

cpp.PointerType

Using the @:cpp.PointerType metadata on an extern class will cause it to be treated as a pointer, this metadata supports the same fields as the above value type one.

Extern classes annotated with the pointer type metadata cannot have constructors as they are designed to be used with the common C/C++ pattern of free functions which allocate and free some sort of opaque pointer, or the pointer to pointer pattern.

E.g. the following native API could be externed and used as the following.

namespace foo {
	struct bar {};

	bar* allocate_bar();
	void free_bar(bar*);

	struct baz {};

	void allocate_baz(baz** pBaz);
	void free_baz(baz* baz);
}

@:semantics(reference)
@:cpp.PointerType({ type : 'bar', namespace : [ 'foo' ] })
extern class Bar {}

@:semantics(reference)
@:cpp.PointerType({ type : 'baz', namespace : [ 'foo' ] })
extern class Baz {}

extern class Api {
	@:native('allocate_bar')
	function allocateBar():Bar;

	@:native('free_bar')
	function freeBar(b:Bar):Void;

	@:native('allocate_baz')
	function allocateBaz(b:haxe.extern.AsVar<Baz>):Void;

	@:native('free_baz')
	function freeBaz(b:Baz):Void;
}

function main() {
	final bar = Api.allocateBar();

	Api.freeBar(bar);

	final baz : Baz = null;

	Api.allocBaz(baz);
	Api.freeBaz(baz);
}

The pointer to pointer pattern which is pretty common is quite difficult to extern without custom C++ glue code, but the new pointer type externs understand this pattern and can be converted to a double pointer of the underlying type as well as a pointer to a void pointer which is also seen in many places.

Internally pointer types and value types are treated almost identically so most of the previous points apply here as well, the main exceptions being that promoted pointers don't have finalisers assigned and that null is always an allowed value.

cpp.ManagedType

When you want to extern a custom hx::Object subclass then this is the metadata to use as it ensures the write barriers are generated for the generational gc. Like the above two metadata it supports the type and namespace fields.

namespace foo {
	struct bar : public ::hx::Object {};
}

@:cpp.ManagedType({ type : 'bar', namespace : [ 'foo' ] })
extern class Bar {
	function new() : Void;
}

In the above sample Bar will be generated as ::hx::ObjectPtr<bar> in most cases.

There is one extra field to the managed type, flags, which is expected to be an array of identifiers and currently there is one flag, StandardNaming. If in C++ you use the hxcpp class declaration macro to create a custom subclass with the same naming scheme as haxe generated classes then this flag is designed for that.

HX_DECLARE_CLASS1(foo, Bar)

namespace foo {
	struct Bar_obj : public ::hx::Object {};
}

@:cpp.ManagedType({ type : 'Bar', namespace : [ 'foo' ], flags : [ StandardNaming ] })
extern class Bar {
	function new() : Void;
}

In the above case Bar will be used instead of the manual ::hx::ObjectPtr wrapping but Bar_obj will be used when constructing the class.

Implementation Details and Rational

Marshalling State

Value and pointer types are represented by the new TCppNativeMarshalType enum which can be in one of three states, Stack, Promoted, or Reference. This is the key to working around optimiser issues, capturing, and some interop features. All TCppNativeMarshalType fields are given the promoted state and TVars can be given any three of the states. Any TLocal to a native marshal type is given the reference state. How TVars are given their state is important, variables allocated by the compiler are given the reference state, only variables typed by the user are given one of the stack (uncaptured) or promoted (captured or nullable) state. This means we dodge the issue with cpp.Struct where you could be operating on a copy due to compiler created variables.

TLocals of the reference state are generated with the new cpp::marshal::Reference<T> type which holds a pointer to a specific type and is responsible for any copying, promotion, checking, and just about everything. For the value type case it's T will be a pointer to the value type, and for pointer types will be a pointer to the pointer.

Semantics

You are required to put the @:semantics(reference) metadata on an extern class when using the value or pointer type metadata, this does feel like a bit of a bodge... I was initially hoping that the value semantic would be what was needed, but tests start to fail when the analyser is enabled with value semantics. Maybe I'm just misinterpreting what these semantics are actually used for. With the reference semantics the tests do pass with the analyser, but from a quick glace that appears to be because many optimisations are disabled on types with that semantic meta...

Compiler Error Numbers

There are several errors you may now get if you try and do things wrong (invalid meta expression) or which are not supported (function closures) instead of vague C++ errors. In these cases I've given them distinct error numbers in the CPPXXXX style, similar to MSVC and C# errors. I plan on documenting these since they're things users might naturally cause as opposed to internal bugs, so I thought it might be nice to give then concrete numbers for easier searching.

Scoped Metadata

I can never remember the names of the C++ specific metadata and end up sifting through the dropdown list every time, so I decided to prefix these ones with cpp. to make it easier.

Metadata Style

I wanted to avoid re-using the @:native metadata for the extern classes as its already pretty common to do stuff like @:native("::cpp::Struct<FooStruct>") so by having a type and namespace field I wanted to make it clear it should be just the type, nothing else. Also with this we can prefix :: to the type / namespace combo to avoid any relative namespace resolution issues.

Eager Promotion

Due to the very dynamic nature of hxcpp's function arguments and return types there are many places where value types which could be on the stack have to be promoted to satisfy the dynamic handling. With my callable type PR this should be solvable.

Future Stuff

Closures

Currently trying to use a function closure of a value or pointer type will result in a compilation error, but now that the variable promotion stuff is in place it should be possible to generate closures which capture the object to support this. Again I wouldn't want to do this until the callable change is in since that will greatly simplify things.

Arrays and Vectors

Value types stored in contains such as arrays are in their promoted state, not a contiguous chunk of memory which I originally wanted. Preserving C++ copying / construction semantics with haxe's resizable array looked to be a massive pain so I decided not to. I do think having haxe.ds.Vectors of value types be contiguous should be possible and open up more easier interop possibilities.

Un-dynamicification

Lots of the cpp std haxe types have a Dynamic context object which is then passed into C++ where its cast to a specific type. With the managed type meta we should be able to "un-dynamic" a lot of the standard library implementation.

Closing

I'm sure there's stuff I've missed but this seems to be much more consistent in behaviour and nicer to use than the existing interop types, I've also added a bunch of tests on the hxcpp side to capture all sorts of edge cases I came across. I will also try and write some formal documentation for all this to encourage this over the existing types.