Eval does not release memory/ has a memory leak

[andig]

The following program sample.go triggers an unexpected result

package main

import (
	"fmt"
	"runtime"
	"runtime/debug"
	"time"

	"github.com/traefik/yaegi/interp"
)

func main() {
	vm := interp.New(interp.Options{})
	if _, err := vm.Eval(`var price float64`); err != nil {
		panic(err)
	}

	for {
		if _, err := vm.Eval(`price = 1.0`); err != nil {
			panic(err)
		}

		debug.FreeOSMemory()

		var stats runtime.MemStats
		runtime.ReadMemStats(&stats)
		fmt.Println(stats.Alloc)

		time.Sleep(10 * time.Millisecond)
	}
}

Expected result

// steady memory consumption

Got

// growing memory consumption

Yaegi Version

0.16.1

Additional Notes

Running an Eval() against a VM without allocating new variables leaks memory. If no allocation happens within the VM, the memory profile of the VM should remain stable.

This seems to make the Yaegi VM unsuitable for long-running purposes.

[andig]

It looks as if the memory consumption comes from growing interp.roots (https://github.com/traefik/yaegi/blob/master/interp/ast.go#L938). Is this something that could be eliminated?

[andig]

Unfortunately, this will eventually OOM our application and breakes Yaegi usage for our purpose of long-running application.

[yusufozturk]

@andig we have the same problem. every eval increases the roots. I see that you found a solution but I didn't understand the root cause from your changes. Was it related to imports?

[andig]

I see that you found a solution

Unfortunately not. This is a showstopper for us.

[andig]

@mvertes the growing roots memory is a showstopper for us. Is there anything we could do/ look into to prevent this?

[andig]

Looking at https://marc.vertes.org/yaegi-internals/ I see:

The memory management performed by the interpreter consists of creating a global frame at a new session (the top of the stack), populated with all global values (constants, types, variables and functions). At each new interpreted function call, a new frame is pushed on the stack, containing the values for all the return value, input parameters and local variables of the function.

I'm wondering why this is necessary and if we could prune old frames or if reusing old frames would be possible?