LTable memory saving proposal

[tul]

• [X] GopherLua is a Lua5.1 implementation. You should be familiar with Lua programming language. Have you read Lua 5.1 reference manual carefully?
• [X] GopherLua is a Lua5.1 implementation. In Lua, to keep it simple, it is more important to remove functionalities rather than to add functionalities unlike other languages . If you are going to introduce some new cool functionalities into the GopherLua code base and the functionalities can be implemented by existing APIs, It should be implemented as a library.

Please answer the following before submitting your issue:

1. What version of GopherLua are you using? : 1cd887cd
2. What version of Go are you using? : 1.13.1
3. What operating system and processor architecture are you using? : darwin/amd64
4. What did you do? : N/A
5. What did you expect to see? : N/A
6. What did you see instead? : N/A

I'm always interested in reducing the runtime memory of gopher lua as my use case has thousands of LStates and tens of thousands of LTables per process.

I noticed recently that quite a good memory saving was possible by removing the k2i and keys fields of the LTable for tables which were never iterated - which is the vast majority in my case. I think it could be interesting to only initialize and start updating keys and k2i on the first call to next, so that LTables which are never iterated do not incur this memory overhead.

It would also marginally speed up table update operations for tables which have never had next called on them, although there would be a slight spike on first call to next.

@yuin what do you think? I'm happy to propose a patch for this.

[yuin]

Sorry for late reply.

This is hard decision for me to make. I think your use case is unusual and most gopher-lua users hardly ever have such number of LStates. So almost users probably prefer speed over memory efficiency.

Could you take benchmarks between the current gopher-lua and your patched version?

[epikur-io]

I really like the idea of @tul since it actually has no real downsides (at least it seems so for most of the use cases I can think of right now)

[chyh1990]

@tul see #283 ?

[rhettli]

@yuin there have too much instance of lua state,and i got them in luapool ,so when one lua state instance run over,it will be reused to do another task, and those lua state spend too much memory , are there have a method to release memory but not kill it to run another task and not spend too much memory?

ps just 2 lua state instance spend me over 400MB memory? unlivable.

L := luaPool.Get()
defer L.Put()  -- put it to lua pool when task done.
if err := L.State.DoString(`print("hello")`); err != nil {
    panic(err)
}
-- Here ,add a function to make it can be used for any times.
L.Sate.Relase()
-- or :
L.Sate.ClearLuaMemory()

[tul]

@rhettli I believe you are correct and that there is no publicly exposed way to reset an LState. When a Lua program finishes globals can be extracted from it by LState.GetGlobal() etc and so the variables cannot be automatically reset as the Lua finishes. The large memory you are seeing is likely because of global variables in the finished program and stack space which is held ready for use.

Obviously the cleanest way to reset is to simply create a new LState and Close() the previous one, but I guess you don't want to do that for some reason? Why is that?

[goftpd]

@tul I'm pretty sure they want to use a sync.Pool to avoid as much allocation as they can. I must admit one of the first things I did was look for a reset and couldn't find one.