Memory Leak: C# Objects passed to Lua do not get Garbage Collected until `Lua` is Closed

[notexplosive]

Hi! I've been using NLua for a little while and I'm a big fan generally speaking! However I think I've found a pretty gnarly memory leak that makes NLua basically unusable for me.

If I acquire a LuaFunction in C# and then .Call() it with a parameter. The corresponding Lua seems to take a reference to that parameter and does not let go of it until the Lua is Closed.

A simple example:

lua = new Lua();

// Assume that `myFunction` is defined in lua

var thing = new Thing();
(lua["myFunction"] as LuaFunction).Call(thing); // thing will not get garbage collected until lua is Closed.

The way my project is setup, I spin up one Lua at the start of the game and it persists for the entire game session. I use the above way of calling code as my primary way for C# code to talk to Lua. This means every single time I use Call() I leak a little bit of memory.

Repro

I was able to reproduce this bug with 2 files. Canary.cs and Program.cs with TargetFramework as net6.0

Canary.cs

namespace LuaMemoryTest;

public class Canary
{
    private readonly string _name;
    private int _tweetCount;

    public Canary(string name)
    {
        _name = name;
        if (Canary.InstanceCounts.ContainsKey(name))
        {
            Canary.InstanceCounts[name]++;
        }
        else
        {
            Canary.InstanceCounts[name] = 1;
        }
    }

    private static Dictionary<string, int> InstanceCounts { get; } = new();

    ~Canary()
    {
        Canary.InstanceCounts[_name]--;
    }

    public static void PrintStatus()
    {
        foreach (var pair in Canary.InstanceCounts)
        {
            Console.WriteLine($"Number of \"{pair.Key}\" Canaries: {pair.Value}");
        }
    }
    
    public void Tweet()
    {
        // Does nothing meaningful, I just wanted there to be some "work" happening in this method
        _tweetCount++;
    }
}

Program.cs

using LuaMemoryTest;
using NLua;

// Sanity check: Create a Canary on the stack and do nothing with it. (GC will clean it up later)
for (int i = 0; i < 5000; i++)
{
    new Canary("C# Stack from For Loop");
}

var lua = new Lua();
lua.DoString("myFunction = function(canary) canary:Tweet() end");
lua.DoString("myTable = { memberFunction = function(canary) canary:Tweet() end }");

// Call myTable.memberFunction() and pass in a new Canary each time
for (int i = 0; i < 5000; i++)
{
    ((lua["myTable"] as LuaTable)!["memberFunction"] as LuaFunction)!.Call(new Canary("Passed to Lua Table Function"));
}

// Call myFunction() and pass in a new Canary each time
for (int i = 0; i < 5000; i++)
{
    (lua["myFunction"] as LuaFunction)!.Call(new Canary("Passed to Lua Global Function"));
}

// Full GC
GC.Collect(GC.MaxGeneration);
GC.WaitForPendingFinalizers();

Console.WriteLine("BEFORE DISPOSE");
Canary.PrintStatus();

lua.Close();
GC.Collect(GC.MaxGeneration);
GC.WaitForPendingFinalizers();

Console.WriteLine("AFTER DISPOSE");
Canary.PrintStatus();

Output:

BEFORE DISPOSE
Number of "C# Stack from For Loop" Canaries: 1
Number of "Passed to Lua Table Function" Canaries: 904
Number of "Passed to Lua Global Function" Canaries: 5000
AFTER DISPOSE
Number of "C# Stack from For Loop" Canaries: 1
Number of "Passed to Lua Table Function" Canaries: 1
Number of "Passed to Lua Global Function" Canaries: 1

Notes

• Even in the best case I end up with 1 of each Canary. I don't know if this is a bug in my reference counter code or some quirk of how the GC works. Regardless, I think the 5000 Canaries proves there's something weird happening here.
• The exact number of Canaries varies from run to run, but not by much (on the order of + or - 5 Canaries on my machine)
• Swapping the order of the "Table Function" and "Global Function" loops yields different results (still high numbers but not necessarily 5000)
• Even if we don't call canary:Tweet() we get similar (but not identical?) results.
• I'm on dotnet version 8.0.101, with TargetFramework net6.0

[notexplosive]

Small update! I found that if I used lua.State.GarbageCollector(LuaGC.Collect, 0); the Lua runtime would let go of all the parameter objects. This is an acceptable stopgap for me.

[juharris]

Thanks for sharing @notexplosive Can you confirm where you put lua.State.GarbageCollector(LuaGC.Collect, 0); to make it work? It's not clear to me if that line configures GC (so it can be done after calling new Lua()) or starts GC and should be done after operations.