cmd/yaegi: fails to import "gonum.org/v1/gonum/mat"

[kortschak]

Importing "gonum.org/v1/gonum/mat" fails.

~ $ yaegi
> import "gonum.org/v1/gonum/mat"
.../src/gonum.org/v1/gonum/internal/math32/math.go:19:21: missing support for type uint: 3

The relevant line in math.go:

	mask    = 0x7f8 >> 3

[ldez]

Side note: gonum contains C bindings which is not supported by yaegi https://github.com/containous/yaegi#limitations

[kortschak]

This is not true. There is no C binding in gonum.org/v1/gonum/mat that is not guarded by a build tag, and only is used by testing when that build tag is present.

[ldez]

It's a side note for us, not related to your issue, don't be afraid.

I did not know how to note it in the issue so I referenced the limitations, but there is no problem.

[kortschak]

That's fine, just noting that it's not true. An import of gonum.org/v1/gonum won't cause any Cgo code to be built unless the user wants that.

[kortschak]

Related though, there is asm that is not guarded in the default case. The pure Go code is built when either the 'safe'/'appengine' or 'noasm' tags are present. Is there a way to specify build tags if asm is going to be a problem?

[ldez]

For now, there a no way to specify build constraints to the interpreter.

[kortschak]

So the follow-on then is whether asm in the source will be a problem.

[ldez]

Could you open another issue about this subject?

[mvertes]

Hello @kortschak, thank you for your feedback and your interest in yaegi, and congrats for gonum, which is really an impressive piece of software, of massive importance.

The issue you have raised here is totally legitimate, and the following discussion about handling custom build tags too. There is no question that this must be fixed in yaegi (and will).

But when those issues will be fixed (and maybe others, lurking behind), we will still miss the point. You will end up being able to import gonum in yaegi, but interpreted from source. Much slower than compiled go, not even speaking about optimized assembly.

The best way to use gonum from the interpreter is to import its compiled binary form (possibly optimized with assembly and/or C), rather than its source form. This is exactly how we provide acess to the standard Go library in yaegi.

The gonum library must be compiled in the executable, and wrapped into a map of reflect.Value symbols (using goexports), and then the symbols are exposed to the interpreter with the Use() method. An import of a gonum package in the interpreted script will then result to access to optimized binary form. As a side benefit, you would even not have to wait for the bug fixes related here.

We are still lacking documentation covering that process, and goexports may need work to be used on packages other than standard library. Sorry for that. And sorry @ldez for hijacking this thread with this huge off-topic digression 😄

[kortschak]

@mvertes Yes, I realised that after looking through the behaviour, hence this PR. I'm interested though in being able to use Gonum via the repl (or something along those lines - for a scriptable graph analysis tool/environment). Here we'd absolutely want compiled Gonum code. Is that achievable?

Answering my own question: yes, by making a copy of the cmd/yaegi and adding the Use call for the gonum/exp/yaegi.Symbols.

[mvertes]

Yes, as you said.

[sbinet]

I vaguely remember discussing this with you @mvertes when yaegi was still called dyngo, but a possible alternative (that was used for Neugram, eg: here) is to use plugin to (compile and then) load external symbols.

this would obviously only work for linux (with some bugs) and darwin (with some more bugs), but could ease the import+use of compiled packages (w/o requiring pre-compiled symbols.)

just saying. :)

[mvertes]

@sbinet, I understand your demand, but we do not plan to use plugin in yaegi/interp for the moment, due to portability and stability issues. This is something we may revisit if the situation improves in Go plugins, but frankly I do not expect much on that topic. In the main use case on our side, we would not even have a Go toolchain available on targets, and that would open too many issues.

Not against experiments outside the interp package itself, at executable level, if one feels the need to do so, and open to insert some hooks in the import workflow to allow that if necessary.

[sbinet]

fair enough :)

[kortschak]

An update on this; while the import still fails, it fails for a sensible reason now, the inability to use the asm implementations in the imported code. This will likely be fixed by #372.

[mvertes]

Just tested the following with yaegi-v0.14.2:

sample.go:

// yaegi:tags safe
package main

import (
        "fmt"

        "gonum.org/v1/gonum/mat"
)

func main() {
        var c mat.Dense
        fmt.Println(c)
}

$ yaegi ./sample.go
{{0 0 [] 0} 0 0}

So I close this old issue. Please feel free to open new ones if more complex cases are failing.