macOS: external drag and drop support

[StarHack]

Adds some very basic DnD support for external files on macOS.

• I don't understand event routing yet, so the implementation is probably incorrect.
• Support for Windows and Linux missing
• It probably should be possible to add this for specific widgets and not for the whole window.

Usage

go.mod

module main

go 1.19

replace gioui.org => github.com/StarHack/gio v0.0.0-20230421195057-eb440387490a

require gioui.org v0.0.0-20230414223051-b6e0376ad2fe

require (
	gioui.org/cpu v0.0.0-20210817075930-8d6a761490d2 // indirect
	gioui.org/shader v1.0.6 // indirect
	github.com/benoitkugler/textlayout v0.3.0 // indirect
	github.com/go-text/typesetting v0.0.0-20221214153724-0399769901d5 // indirect
	golang.org/x/exp v0.0.0-20221012211006-4de253d81b95 // indirect
	golang.org/x/exp/shiny v0.0.0-20220827204233-334a2380cb91 // indirect
	golang.org/x/image v0.0.0-20220722155232-062f8c9fd539 // indirect
	golang.org/x/sys v0.0.0-20220825204002-c680a09ffe64 // indirect
	golang.org/x/text v0.7.0 // indirect
)

main.go

package main

import (
	"fmt"
	"image"
	_ "image/gif"
	"io"
	"os"

	"gioui.org/app"
	"gioui.org/io/system"
	"gioui.org/io/transfer"
	"gioui.org/layout"
	"gioui.org/op"
	"gioui.org/op/paint"
	"gioui.org/widget"
)

func loadImage(file io.ReadCloser) (widget.Image, error) {
	img := widget.Image{}
	decImg, _, err := image.Decode(file)
	if err != nil {
		return img, err
	}
	img.Src = paint.NewImageOp(decImg)
	return img, nil
}

func main() {
	go func() {
		w := app.NewWindow()
		var ops op.Ops

		f, _ := os.Open("img/placeholder.png")
		img, _ := loadImage(f)

		tag := new(int)

		for {
			e := <-w.Events()

			switch e := e.(type) {
			case system.DestroyEvent:
				return
			case system.FrameEvent:
				gtx := layout.NewContext(&ops, e)

				transfer.TargetOp{
					Tag:  tag,
					Type: "image/png",
				}.Add(&ops)

				for _, gtxEvent := range gtx.Events(tag) {
					switch e := gtxEvent.(type) {
					case transfer.DataEvent:
						file, err := e.Open()
						newImg, err := loadImage(file)
						if err == nil {
							img = newImg
						} else {
							fmt.Println(err)
						}
					}
				}

				img.Layout(gtx)
				e.Frame(gtx.Ops)
			}
		}
	}()
	app.Main()
}

Demo

https://user-images.githubusercontent.com/5388534/221357178-7e800dc7-9513-4575-9e84-1b191a6cb406.mov

[StarHack]

@eliasnaur Updated implementation to use io/transfer package instead and took into account the other change requests to simplify the implementation. Also updated the usage example above.

[StarHack]

@eliasnaur Many thanks for your feedback. I have adjusted the implementation which now allows to register a TargetOp with the desired mime type and the event is now routed to the appropriate tag. Furthermore, the sample implementation above has been updated and all commits have been squashed into one.

[fjl]

@StarHack thanks for working on this! I really need it in my own app.

[StarHack]

@eliasnaur @fjl Many thanks for your feedback! I appreciate it. I changed the implementation to return the result of os.Open() instead and we now only send the event to the topmost element of the hitlist.

[whereswaldon]

Just a thought:

So, overall there is lots of inspiration to be found in the way browsers handle it. For obvious security-related reasons, the browser does not provide access to the full file path (anymore). However, the basename of the file is there.

I think that gio should provide the full path, or at the very least the file basename, alongside the open function.

Gio does run in the browser when compiled to WASM, so whatever we do needs to be implementable within the browser. I suppose we could degrade gracefully from full OS paths to whatever the browser offers though, so long as we document that you may get a full OS file path.

[eliasnaur]

@fjl adding all that information seems a bit much to promise. Questions:

• Can a list of dropped files be implemented as separate DataEvents?
• Are you ok with just an URL field with a file: reference for local files?

[StarHack]

@eliasnaur Do you have further feedback on the existing implementation? I'd like to finalize this pull request at one point or another and add support for Linux and Windows as soon as I've time to do so.

[fjl]

I proposed fs.FileInfo mostly because it is the usual way of representing 'information about a file' in Go. If you feel it's too much, of course I'd also be fine with just a name/URL. If the URL is a complete path, the app can always resolve all necessary information itself. There are two situations where it will not work: (1) with the WASM/browser backend, (2) when the app is running in a sandbox. But we can worry about those cases later.

Delivering multiple files as individual events is not great, but this could just be my personal opinion. I have no experience to back that up. The act of dropping a set of files is 'one event', logically speaking. In some contexts, dropping multiple files will make sense, and requires handling from the app. This can be dealt with by accumulating events. In other cases, e.g. 'avatar image box' in a messenger, only a single file is acceptable. If multiple files are dropped, the app will process every file (in case of image: open, parse, crop etc.) and then use the last one. It can't do anything else because it has no knowledge about multiple drop events being in the queue.

[fjl]

I don't want to be the guy holding up this extremely useful PR. If you feel it is the simplest, let's just go with a URL field on transfer.DataEvent and worry about the other problems later!