Proposal: Runtime dynamic linking

[diamondburned]

Reasoning

• Much faster compile time, HUGE!
• Support multiple GTK minor versions
  • Allow program developer to determine the minimum version, not gotk4
  • Allow program developer to easily support multiple versions

Approaches

• libgirepository
• ~~GModule~~
• Small wrapper around dlfcn.h/syscall, package pkg/core/dlsym maybe?

Implementation

libgirepository

https://gi.readthedocs.io/en/latest/writingbindings/libgirepository.html

GModule

package gmodule

type Module struct {
    syms   sync.Map
    handle uintptr
}

func MustLoad(name string, flags Flags) *Module

func (m *Module) Symbol(name string, v *unsafe.Pointer) error
func (m *Module) MustSymbol(name string, v *unsafe.Pointer)

///

package gtk

var module = gmodcore.MustLoad("???")

func NewWidget() *Widget {
	var _cfunc unsafe.Pointer
	module.MustLoad("???", &_cfunc)

	w := (*(*C._gotk4_gtk4_new_widget)(_cfunc))()
	return cast(w)
}

dlfcn

package dlsym

type Handle struct{}

func MustOpen()

func (h *Handle) Symbol()

// TBD

Considerations

• Give each function a global func_NAME uintptr
• Consider atomic.CompareAndSwapUintptr to properly do this in a thread-safe manner

[diamondburned]

Further Thoughts

• libgirepository requires usage of GClosure, which we already have before but deprecated because it doesn't handle specific scenarios e.g. a pointer-length array parameter pair. We can code-generate that, but is there a point?
  • Maybe we don't need to export the symbols, sure, but that wouldn't help anything.
  • Probably consider this as the last resort.
  • If we have code to deduplicate type conversions, that could be worth doing, but if we expect conversion routines to be inlined, then there's no point for us to not inline them ourselves.
    • Maybe there are certain things not worth inlining?
  • GCClosure, the current method, has to go through GValue-conversion anyway. It might be better if we just generate the same function trampoline that decodes a list of GValues differently then invoke the function directly.
    • This is the code-gen way.
    • This would work far better than the old GClosure method. Performance might be roughly the same.
    • We would have to know what GValue methods to use, which is simple enough but takes fiddling.
• A dlsym wrapper seems like the better option here(?).

[diamondburned]

WIP branch at https://github.com/diamondburned/gotk4-libgirepository.

I'm unsure if gio should be runtime-linked or compile-time-linked. It takes ~1 minute to build it with cmd/cgo.

[diamondburned]

Error log:

# github.com/diamondburned/gotk4/pkg/atk
pkg/atk/atkcomponent.go:142:7: could not determine kind of name for C.AtkRectangle

// NewRectangle creates a new Rectangle instance from the given
// fields. Beware that this function allocates on the Go heap; be careful
// when using it!
func NewRectangle(x, y, width, height int) Rectangle {
	var f0 C.gint // out
	f0 = C.gint(x)
	var f1 C.gint // out
	f1 = C.gint(y)
	var f2 C.gint // out
	f2 = C.gint(width)
	var f3 C.gint // out
	f3 = C.gint(height)

	v := C.AtkRectangle{
		x:      f0,
		y:      f1,
		width:  f2,
		height: f3,
	}

	return *(*Rectangle)(gextras.NewStructNative(unsafe.Pointer(&v)))
}

Fix is to do var b [n]byte and use unsafe.Add(&b[0]). Go will allocate b on the Go heap for us.

Alternative solution

Fix is to have Go generate a Go struct like so:

type rectangleNative struct {
    x      C.gint
    y      C.gint
    width  C.gint
    height C.gint
}

Then we can generate unsafe.Offsetof to assert that the offset of these fields match what's declared in the GIR file. Since unsafe.Offsetof is a constant, an if unsafe.Offsetof(x) == N check can easily be omitted.

Cons:

• What should we do with nested records? field [n]byte maybe? That would be weird.

[jwijenbergh]

Sorry for the noise, but maybe https://github.com/ebitengine/purego could in theory be used for dynamic linking? They have some dlfcn stuff, hopefully that helps

[mgord9518]

Would this make it possible to have fully static binaries?

[LukeShu]

@mgord9518

Would this make it possible to have fully static binaries?

I don't think so. The binaries wouldn't need to link to libgtk and friends anymore, but they'd still need to link to libdl. (Perhaps that's avoidable with https://github.com/ebitengine/purego though?)

[jwijenbergh]

@mgord9518

Would this make it possible to have fully static binaries?

I don't think so. The binaries wouldn't need to link to libgtk and friends anymore, but they'd still need to link to libdl. (Perhaps that's avoidable with https://github.com/ebitengine/purego though?)

With purego it's also complicated https://github.com/ebitengine/purego/issues/132

[diamondburned]

This proposal doesn't aim to produce fully static binaries, FWIW. My goal was to link to some very basic core libraries (i.e. glib and gobject-introspection), and then link to everything else during runtime.

This proposal has less to do with producing a pure Go binary or a static binary and a lot more to do with optimizing for build times. Implementing this proposal would allow some of the much larger packages (e.g. GTK4) to be in pure Go (linking to already-compiled Cgo packages), speeding up compile times significantly.

[T0astBread]

Sorry for the noob question but is there anything to try out in the 4-libgirepository branch at this point? And if so, does it require any special build flags? (I tried just replacing the regular version with that branch but build times were about the same.)

[diamondburned]

I don't think the runtime linking mode was ever completed or even reached a runnable state. I believe all changes from 4-libgirepository should've already been merged into 4 guarded behind an environment variable.