LPCGame
LPCGame copied to clipboard
Twinklebear
Lua Embedding
Implement Lua for scripting objects behavior
- Need to determine what is necessary to be exposed and what should be handled internally and how the API should work and such
- Should entities be drawn automatically? Or should it be required to make the call yourself in draw?
- I think an image file should be described in the entity's json file and used. Rendering should be toggle-able to enable/disable drawing of the image
Need a better way for Lua scripts to load the modules they need through something like require or such
- Need a system to prevent scripts from being able to try and register the same module twice as it causes a crash. This issue is easy to encounter if perhaps you call "dofile" from a script with a module loaded and then the script that's just been loaded via "dofile" also attempts to load the same module, it causes a crash. Need to track which modules are registered by the state and quietly ignore re-registration attempts.
How should the State class be handled?
- If I create the ability to call state functions like SetExit from Lua, will ObjectButton no longer be needed? since Button's OnClick script would then simply call the function? I'll try this and then decide what to do with ObjectButton.
- I think I've got a method for this setup, see the static StateManager function ChangeScene
- This isn't quite right. The button needs to set exit on the actively running scene so that it can exit cleanly
- I think I've got a method for this setup, see the static StateManager function ChangeScene
- Will need some way to grab the active state, perhaps from StateManager? It's a pure-static class so yes, I think that will work
- Also brought up by this is GameState vs. MenuState, ie. will such a distinction be needed or even a good idea once so much of the behavior is defined in Lua? I think they should be merged together when Lua entities fully take over, leaving us with a base State class that's used for the game and the EditorState which will only be used internally for creating the editor itself.
Should states be run via script? Hmm. What to do with states..
- States will execute their basic C++ functionality, such as making the calls to the manager, and other background stuff, but will also call a function on a state script if one is attached
How to save data from script?
- Have decided to leave this up to the user, so editor created data will be handled by JsonCPP and script data will be handled by whatever is desired for Lua, and will have no interoperation between the C++ saved data and custom script data.
Helpful Reading Material
Programming in Lua: The C API Luabind Documentation
Current state of Lua work:
- LuaScript class manages running of a Lua state (ie. script)
- Entity class calls functions in an attached Lua script
- Have adding my own functions to Lua's package.loaders table to be used in performing module lookups. Will also enable loading of engine scripts via calling require "scripts/scriptname.lua" and it will append the correct relative path to the filename so Lua's dofile can find it.
Next work will be to re-work Button and ObjectButton to perhaps just be scripted entities. Or maybe Button can remain but be a default Button configured Entity for extension by the user.
Thinking about creating a statemanager lookup system, that will enable entites to perform function calls on any part of the active state or statemanager.
Would be possible to call a few general StateManager functions that would influence the active state, such as setting exit, or attempt to call a Lua function. The Lua function call should allow for some guidance of where it should look, ie. State or Entites. Then will descend into the appropriate area, if Entity, find the entity by some name and try to call the function name passed on its Lua script.
The difficulties in setting this up is that it's not possible to just load up the lua script file we want and call the function because it won't be in the context of the active lua state we're thinking of, but rather the script will be loaded into the lua state doing the loading, so it won't have the desired effect.
Leads me to the StateManager based lookup system, as the StateManager is static and knows which State is running, which in turn knows its EntityManager and so forth, which enables us to perform a call into the lua state we want (on that's already running) through something that we can easily access. At least this is the idea. Implementation is yet to be done. This is more a note so I remember the idea.
My commit from this evening:
- Added OnClick functionality to Entity so that it can be used via Entity script
- Added file Entity loading support for menustate to lead Entity based buttons
- Added a test button Entity that's scripted via Lua (quitbutton.lua) and described in quitbutton.json
- Tweaked the Text::Size(w, h) query function to now take pointer, also added Text::W and Text::H to directly get the Text width and height
- Considering adding active clip setting/usage to Image same as is done in AnimatedImage at the moment
- Currently no ability to effect other areas of the State besides simply setting an exit through the StateManager, but that's coming soon.
Will be working on a lookup system next to dispatch function calls to the active state or to entities within the state.
Idea for lookup system based on taking a peek at unity:
- Entities: Can be looked up via some function by name or tag and will return an array of Entity pointers to all Entities with that tag or name
- States: Perhaps just return the State pointer to call the function on
However this still doesn't solve the issue of passing varying parameters if the call goes through C++. Is there a way to bypass C++ entirely can call a function directly on the Entity's Lua state? Or is there a way to do it through C++?
edit: What if i pushed the desired Entity function onto the lua stack to be called?
Some notes to myself: pushing data between lua states lua_xmove and see lua_pcall for calling functions. and generic call function
but will this resolve the issue of handling varying/multiple arguments that are potentially of type that I've defined instead of a default type? How are my types represented in lua so that I know how to push and get them? Do i need to worry about this if i use lua_xmove to push data back and forth?
Call could be something like callfunc(nargs, nreturns, args) is it possible to deduce the # of args through the call without having to have nargs?
Can now lookup LuaScripts by name, get their LuaScript instance pointer and call functions on them. These functions should take no parameters and return nothing, will be trying params/returns now. Unsure of whether or not lua_xmove will actually be the right function to transfer things as it mentions they should be members of the same global lua_state.
These changes are in the experiments branch, in the LuaCrossStateCallTest project
It seems the issue is that data I want isn't pushed onto the stack. Performing a stack dump on scriptB at the time of calling script:CallFunction(..) from scriptB:
require "LuaScript"
require "Vector2"
print "Script B!"
script = LPC.LuaScript.GetScript("scriptA")
--vect = LPC.Vector2i(5, 4)
test = "Some Test string"
print (test)
script:CallFunction("Test", 2, 5)
gives
Lua Stack Dump: userdata, Test, 2, 5,
so how can i get the variable test? It seems luabind doesn't accept the "..." param, so i can't use a var args list, so I need someway to push an arbitrary amount of data to the stack so that it can be transferred over.
Other idea: define my own generic call function based on the link above.
How will I get the LuaScript class context if I write a version of lua's call_va?
Super slow version is made! However due to some issues with lua_touserdata mangling my data the script has to be looked up by name each call, which is really really bad. Besides that, it appears to work to pass simple params, unsure on userdata. Need to look into return values as well.
Will check out passing userdata and return values and then look into why lua_touserdata mangles my LuaScript*
Return values work, userdata seems to be a bit problematic to pass
side note to self: luacfunctions return # should be the number of return values to expect back from the function, so all the modules register functions should return 0.
It seems that the userdata issue was related to the way the luabind modules defined. I made my own class and registered using the Lua C API and was able to pass it between states as lightuserdata.
Only issue: Because each instance of the class is registered with the metatable that tracks whether or not it's a piece of that userdata an issue arises when passing userdata between states as the metatable context is lost. This leaves two options:
- Use an add function to add the userdata back to the desired metatable so it can be used as normal, in the hopes that you didn't pass some other userdata
- Perform a check on the userdata in the "from" state and register the data with the appropriate metatable in the "to" state, this could potentially be unbelievably slow.
- Have user specify via some shorthand string the types of the userdata being passed, then step through the stack looking for userdata and register the tables accordingly.
I'm unsure if this means that luabind will have to be abandoned and I hope not since i do really like it. Perhaps there's a way to get this working with luabind, although my previous attempts where pretty unsuccessful, even when using unsafe methods for casting userdata back to its expected type.
Maybe if we dump luabind we can use LuaJIT? It's gonna hurt though to migrate everything over.
Overloaded C++ functions? Maybe check size of stack and compare to num params each is expecting?
Added a note about proving that we're calling into the existing state, although the usage of global someNum doesn't really prove anything. The fact that Script A! only prints once, proves that we're calling into the existing state. But i also wanted to try using the state's data as well.
Interesting developments in the Lua C API work. I've decided to attempt to re-make the LuaScript class with the Lua C API as LuaCScript (see experiments branch)
LuaCScript currently deals with double pointers because the TScriptMap is a map<string, LuaCScript*> so when you want to retrieve one to perhaps call a function on or get data, you get a LuaCScript** so that you can point to the pointer. Haven't yet put together the LuaCScript version of GenericCall but will work on that next.
I also need to figure out usage of constructors and destructors for userdata that will be GC'd, such as the LuaRect. Non-GC'd object will be used as double-pointers in Lua and memory managed via C++ while GC'd objects will be used as single-pointers.
Once I learn how to do everything that luabind currently hides I'll drop it and begin using the Lua C API, and look into making use of LuaJIT to boost speed.
I've decided to go with option 3 for now:
- Have user specify via some shorthand string the types of the userdata being passed, then step through the stack looking for userdata and register the tables accordingly.
I've written the generic call function for LuaCScript, next I need to write a parser for the udata signature that will step through the stack and at each udata perform the appropriate metatable registration.
It's assumed that the script being called into has the lib associated with the various udata params open, or else it wouldn't be able to use them anyways, and the metatable registration would fail.
Should also do a check to determine if a udata type list was given, ie. after popping the script udata, function name, # params and # results off the stack, if a udata type string was given the stack size should be # params + 1. If no udata type list given, check through the stack for udata. If any udata is found, print a warning about the speed impact of trying to resolve its type and register it appropriately and then attempt to resolve the udata type.
Is there a way to find the metatable some udata is in just from the udata itself?
Have decided to leave luabind in favor of the Lua C API. This is a list of things left to test in the experiments branch before migrating master over:
- The above comment about udata type "safety/checking"
- Overloaded operators, see Relational Metamethods and Metatables
- Constructors and destructors for types whose memory will be fully managed by Lua, ie. single pointers types, double pointers are managed by C++
- Property accessors, ie. the current method for getting a vector's x value is vector.x, currently only have vector:getX() available in LuaCrossStateCallTest
- Enums ie. KEY_W
- Other things I think of?
Working on property accessors for set/get type behavior through
--Get the value of x from r
num = r.x
--Set the value of x
r.x = 5
I've figured a workaround for __newindex where instead of registering a table of functions i register a lookup function which will determine the appropriate setter and call it.
Contemplating doing the same for getting where if only the index val was passed it would return the value at that point, however I think I could push a table of functions to __newindex and use those, but I'm not sure. I'll check back at Lua IRC later but will fully implement the crummy method for now to test it.
Wrote an accessor lookup table which is registered in __newindex and works for sets but not for gets, ie. with print(r.x) for ex. it instead prints a function address of some sort using it as :accessor("x") works, but i think it should be in __index to be a getter, but I need the other functions in there as well. Need to work more on this.
What I really need is to push tables to __index and __newindex but I'm unsure how to do this. Back to IRC tomorrow. Will work on udata parsing for now
Addendum to udata signature idea: This may not be necessary, it looks like I can snag the metatable of objects on the stack, so i can define a function that all udata will have in its metatable like mytype or something, grab the metatable from the udata, call that function and check the result and use the result to select the appropriate metatable for the data to be registered with in the target state.
See this SO post
This worked well. Now udata's metatable has a type function which will return the C++ classname as a string, so in my call function i can check udata for this field and resolve the type and thus determine the correct metatable to register the type with in the recieving state.
Have made a primitive version of the system which resolves the udata type and will add it to the appropriate table. Currently only for LuaRect, but generic version coming soon. udata signature string is not needed, so is removed.
Generic version will step through the stack looking for udata, when it finds it it will perform readType call on the udata to get the typename, and then use this typename to get the appropriate addX function to add the correct metatable in the receiving state so that the data can be passed seamlessly. (User defined types??? i hope not.. maybe there's a way, but look at much much later)
Current Progress:
-
udata type "safety/checking" when transferring between states This can be done, I stick a function "type" onto the table which returns the C++ classname of the object as a string, so I know the appropriate type to register in the recieving state. What's left to be done is create a lookup table to check through, and make the program run through the stack and query all userdata params/results being passed between states and perform the appropriate registration.
-
Overloaded operators A bit unsure on this, although i do have my LuaRect's less than, equality and less or equal operators done, will be testing addition and so on soon. It seems that I can only use operators on types that are the same? or define the same __operator function? I'm not sure, will have to look into this.
-
Constructors and destructors for types whose memory will be fully managed by Lua, ie. single pointers types, double pointers are managed by C++ Constructor syntax is now usable by creation of the __call method on the class metatable, ie. for LuaRect you do
r = LuaRect(1, 2, 3, 4) -
Property accessors, ie. get/set It seems that I can only get setters working via __newindex and call with r.x = 4, however i can't seem to make getters work through the same syntax. Unsure if it's worth focusing more on this at the loss of working on other areas, will leave for now. You get with r:x()
-
Enums ie. KEY_W This is easy, if a bit tedious, we just push the enum value and then set the field on the class metatable to enable access via LuaRect.ENUM
Progress update:
- UserData type preservation when transferring data between states In place. LuaScript will hold a map of the various add to table functions for each lib and will step through the stack and mark the userdata and then once data is on the target stack, run through again and perform the table registrations
- Operators In place. Tested very thoroughly with Vector2f lib
- Constructors & Destructors In place. Constructors are made by setting the newX function as the index __call on the class metatable (NOT the object metatable, ie. the one with __index) Destructors refer to Lua's __gc method, which is placed on the object's metatable.
- Property accessors In place. The set lookup table through the accessor function works well through __newindex. I'm unsure how LuaBind made r.x also function to get the value. Maybe they directly stuck the getter onto the object metatable, so calling r.x returns the value of x? Hmm. Is it worth bothering with this? r:x() for getting is fine..
- Enums In place. See the example of LuaRect.ENUM being put on the metatable. It's a bit tedious to type out, but it was tedious in Luabind too, so i guess there's not much to be done to improve it.
The migration of master over to the Lua C API is going to take a long time, and longer to fully test and tweak. In addition I have to put in the Lua -> C++ call dispatcher, ie. calling into State functions. However this shouldn't be used for a lot I think, as the majority of user defined stuff should be written in Lua scripts running on Entity's which can be snagged and called Lua -> Lua. Anyways, gonna take some time to get this in.
As an implementation note: The majority of Lua functions will be private besides those that are absolutely necessary to be in C++, and will be defined (in header & implementation) after all C++ members, and will have names beginning with lowercase characters. The hope is to make all Lua specific functions very obvious to try and keep the massive amount of them from getting in the way as much as possible. They should also be easily identifiable by the fact that they all take a lua_State pointer as a param.
Additional note about implementation in the main engine: I may bump the Lua C API code into its own class or namespace, so it's separate entirely from the C++ class to avoid clutter.
Decided on this method for the implementation: Because all functions for Lua interaction are static, I've decided to remove them from the C++ class and push everything into its own namespace, LuaC. Within this namespace the libs will be put into classes corresponding to the class they're the library for. So Entity's lib will be LuaC::EntityLib, and so on. The goal here is to separate the Lua C API code from the core engine code, to prevent a lot of clutter and messyness as a result of loading the class up with a ton of functions that are specific to the Lua C API.
Files will be luacentity.h/.cpp and put into a folder titled luac located in the top directory.
Lua good practice note: The stack should be back to its initiali configuration after the function call has finished, not the way i do it now, where i knock everything off the stack.
Note to self: Use std::function for the function pointers, see http://en.cppreference.com/w/cpp/utility/functional/function
cool new C++11 goodies
Additional note: I don't think Lua will accept a std::function, will have to use pointers there. But other internal stuff can use it.
Also note: None of the toString or concat operator functions will be written for the libs until everything else is written, because I need to write std::string conversion operators for all the classes that have Lua Libs. This std::string operator can also serve as a debug dump sort of thing, pushing all relevant information about the object into a string.
There are various classes which should/shouldn't be managed by Lua.
Classes that should be managed by C++ (use double pointers in the libs)
- Entity
- Physics
- States
- Camera
- Image, AnimatedImage, Text?? Unsure
Classes that should be managed by Lua (single ptr)
- Vector2f
- Rectf
- Color
- Other primitives?
Classes that will be very different, will just consist of a single class function table, no object metatables
- Window
- Input
- Statemanager
- Debug
- Math
Method for implementing the very different classes can be tested with the Debug class, as it has no dependencies.
Entity class changes question: Should Update and Move be rolled into one function and should we call Physics::Move(deltaT) before returning from Entity::Update(deltaT)
Work update
Classes that should be managed by C++ (use double pointers in the libs)
- Entity - written but not usable, needs to be able to register itself with the EntityManager
- Physics - written, but needs to be used in context of an entity
- States
- Camera
- Image, AnimatedImage, Text?? Unsure
Classes that should be managed by Lua (single ptr)
- Vector2f - done
- Rectf - done
- Color - done
- Timer - done
Classes that will be very different, will just consist of a single class function table, no object metatables
- Window
- Input
- Statemanager
- Debug - done, see luacdebug.* for how to register the other pure-static classes
- Math
Question: for lua_xmove is the object metatable moved as well? I assumed no because i got "attempt to index userdata" errors, but maybe it just didn't know what to do with the table? I should check for certain.
On the C++ side, some overloaded operators shouldn't be member functions?
regarding my recent commit, what if i got a pointer to the entitymanager instead of operating on it through the state? It crashes when trying to call the entitymanager functions, so what would happen if i got it directly?
Decided to get EntityManager shared_ptr directly, now also got State and Entity libs working along with cross lua-state calls, which does still need to be tested with user data params and return values.