Jumper
Jumper copied to clipboard
Yonaba
Uses way too much memory.
I use Jumper, astar, for river generation and it uses way too much memory. I was wondering if its possible if you could do some internal optimization for memory. Some suggestions are allowing the user to use an ffi struct for the map data or if you didn't use a class for the nodes internally. Tables take up a large amount of memory so using a class for nodes makes it take up more memory than needed.
Hi @Bobbyjoness, Thanks using Jumper. And well, thanks for bringing this up. Actually, you are somehow right. Jumper uses memory in order to represent the grid and nodes. And well, the amount of memory being used scales (exponentially...?) with the map size.
Thing is, I never wanted to dig much more in solutions to reduce the amount of memory used for one simple reason, it makes the code much more cleaner, and easier to maintain. And very convenient to me, as-is.
But suggestions are welcome. See, I am not at all proficient in LuaJIT and ffi structs (since I haven't used it). Can you please explain in greater details what would be the benefit of offering support for ffit structs ? I might look into this direction, if promising.
FFI structs use way less memory than a lua table. It's basically c structs. A 2d array of structs would use way memory. I'm not sure but maybe even a 2d array of ints/floats can be used. They don't have the overhead of a lua table.
I know its the holidays but if you want I can help work on this issue now and make a PR. I just need to know what route you want to go to optimize the memory. This is really a critical issue for me because currently in my project it is using over 1gb of memory for a line of a distance 2000 with a node at each pixel. This is very important because there is a bug in luajit causing memory to be maxed out at 1gb on Linux and 2 on windows/osx. A good example of ffi use to reduce memory usage can be found here https://github.com/excessive/cpml/blob/refactor/modules/vec2.lua
Just the one line that uses structs makes it use less than half the memory. Possibly the same can be done for the node class
I have been looking at the code for the node class and the way it is used seems messy. This could be just my preference but it does not use getters and setters for all the fields it can have. This is an issue because for me to change the way node works I have to dig through all the code to find out what data the node class holds and how I could optimize it to use less memory.
https://github.com/Bobbyjoness/Jumper/commit/1cc2b5301d93851240e5906c7b93364f8f2f9ae8 I made a commit to my fork that will make the Node object utilize ffi structs. I can not make a PR because I had to break things to get this to work. Namely clearance. I did not know what the clearance table holds so I couldn't figure out the best way to represent it in a struct
@Bobbyjoness I tested your optimization and it does not work reliable. Using the ASTAR algorithm it crashed after the calculation of one path. Using another algorithm just didn't work due to memory access errors.
Error: /libs/jumper/core/node.lua:98: cannot convert 'nil' to 'float' stack traceback: [string "boot.lua"]:637: in function '__newindex' /libs/jumper/core/node.lua:98: in function 'reset' /libs/jumper/pathfinder.lua:359: in function 'reset' /libs/jumper/pathfinder.lua:340: in function 'getPath' /classes/misc/PathFinder.lua:20: in function 'findPath' main.lua:67: in function 'update' [string "boot.lua"]:509: in function <[string "boot.lua"]:493> [C]: in function 'xpcall'
