tomjn
Separate C++ and Lua into 2 repositories
Currently this repo contains a native C++ AI, and the lua component. This won't work going forward with gadget AIs as it means duplication.
So I'd like to separate out the lua AI portion and have it as a submodule, so that it has 1 canonical source, rather than a native AI, and a gadget AI copy that need constant merging
@abma what needs doing at the Spring engine side of things to ensure this continues to work? There are reports that the EvoRTS submodule is working but not the BA/BAR submodules, I imagine changes will be needed for that too? What do you recommend?
what needs doing at the Spring engine side of things to ensure this continues to work?
a stable spring release should be done before changes at the infrastructure delay a release.
- documentation needs to be updated (wiki, ...)
- travis / buildbot scripts needs be fixed
- ...?
basicly not much: i still see me asking myself: why are these lua scripts submodules? wouldn't it be better to directly integrate them into the game and then make shard load it from the game?
A lua gadget AI has bootstrapping code that the native AI doesn't have and vice versa. Coupled with the fact the gadget AI isn't in a 100% state, and that there are advantages and benefits to a native AI, separating out the AI this way simplifies things and avoids a mess On Tue, 21 Jun 2016 at 16:10, abma [email protected] wrote:
what needs doing at the Spring engine side of things to ensure this continues to work?
a stable spring release should be done before changes at the infrastructure delay a release.
- documentation needs to be updated (wiki, ...)
- travis / buildbot scripts needs be fixed
- ...?
basicly not much: i still see me asking myself: why are these lua scripts submodules? wouldn't it be better to directly integrate them into the game and then make shard load it from the game?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Tarendai/Shard/issues/102#issuecomment-227471235, or mute the thread https://github.com/notifications/unsubscribe/AADl55zpYP4Pnh4De4gP3DiefmQ67uaPks5qN_79gaJpZM4I6xY8 .
https://github.com/eronoobos/ShardSpringLua & https://github.com/eronoobos/Balanced-Annihilation-Shard-LuaAI
when the ai is updated a commit has to be made to the game?
(this makes the diff VERY low to having it in the same repo)
I don't follow, but you made me realize that while https://github.com/eronoobos/ShardSpringLua/tree/master/luarules/gadgets/ai contains just the base Lua files, https://github.com/Tarendai/Shard/tree/master/data/ai contains the game configurations, too. Making those two directories one repository would still reduce my headache of manually copying from one to the other, but if a game repository on github wanted to use that repo as a submodule, they would have to make a pull request just to update their own config, even if that config was its own repository. That mess seems worse than the current mess.
I suppose, @Tarendai, the directory structure could be changed to something like ai/base/ and ai/config/, if we changed shard_include? Not sure if that's a good idea or not
I don't follow,
the game depends on a specific ai version: when the ai needs an update, the game has to be updated, right?
in other words:
what needs to be done for the
- ai dev
- game dev
- engine dev
- player for
for updating the ai?
when the ai needs an update, the game has to be updated, right?
yes, of course. but that's true of any Lua AI
if we're talking about the C++ Skirmish AI, then it's the engine that needs to be updated
For the moment the native C++ AI is the prime version of the AI, and the most fully featured. This may change in future, but we have a AI, written in lua, that works with an API, and 2 implementations of that API ( LuaAI and NativeC++Wrapper ).
Bundling the AI and the Native C++ wrapper is causing organisational and structural issues for the development of the Lua AI implementation ( where most of the future interest in the AI lies as far as the spring engine is concerned ). To maintain momentum I'm eager to proceed as soon as possible.
in other words:
what needs to be done for the
ai dev
Remove the AI from this repo and add a submodule to its new location
game dev
At the moment, nothing, that part hasn't been resolved, and isn't relevant to the engine build process or the Shard native C++ API implementation
engine dev
Adjustment of the build process to ensure that all submodules are pulled in during build process, or reassurance that this is already the case
player for
Nothing, the immediate result for them would be the same as before
for updating the ai?
Business as usual
Content devs who want to use the 100% lua implementation of Shard will have to work with a partial API until it's fully implemented, and will need to take the needed steps to set it up in their games, but this has no bearing on the engines build process. Including the necessary parts to make Shard work as a LuaAI inside this repository would mean additional manual steps, complicated processes, and a significantly higher barrier to entry and support burden. Currently the best version of Shard for gadget LuaAI is in a forked repo that could never be merged directly into this repository, duplicating efforts
that doesn't answer my questions: what needs to be done to update (with submodules) in the future when the ai changes for the involved persons?
imo its a lot more work for several people in comparison to having it in one repo.
Other than git sub module update? So long as the engine always pulls the latest sub modules there shouldn't be an issue. My understanding is that this already happens with EvoRTS On Fri, 1 Jul 2016 at 23:16, abma [email protected] wrote:
that doesn't answer my questions: what needs to be done to update (with submodules) in the future when the ai changes for the involved persons?
imo its a lot more work for several people in comparison to having it in one repo.
— You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/Tarendai/Shard/issues/102#issuecomment-230059105, or mute the thread https://github.com/notifications/unsubscribe/AADl53JAKKOYDDROwb_TIn-3HiSiSoDwks5qRZGqgaJpZM4I6xY8 .
in my understanding: ai dev needs to update his repository and send a pull request to shard game dev needs to send a pull request to the ai dev engine dev needs to merge pull requests from shard player needs to update engine ?!
so for an update to take place in the engine, ai dev + shard dev + engine dev needs to merge the pull request.
thats a long chain which will lead to large delays when developing imo and is a lot of overhead.
I'm not sure where these pull requests are coming from, and game developers can release via Rapid and don't require an engine release
The 100% lua AI built with gadgets is irrelevant to this discussion and has no involvement with engine development.
For the native C++ implementation I fail to see how the process is different from an engine perspective. Shard is already synced with the engine releases, I'm not sure how the project is managed internally is relevant.
We have but 1 issue. If Shards C++ implementation uses a submodule, will that be pulled in via the engines build process, and the submodules inside that. Or will we end up with a Shard that complains at startup that no lua files are present? Will it still pull in the EvoRTS and BAR/BA submodules?
On Sat, 2 Jul 2016 at 13:04, abma [email protected] wrote:
in my understanding: ai dev needs to update his repository and send a pull request to shard game dev needs to send a pull request to the ai dev engine dev needs to merge pull requests from shard player needs to update engine ?!
so for an update to take place in the engine, ai dev + shard dev + engine dev needs to merge the pull request.
thats a long chain which will lead to large delays when developing imo and is a lot of overhead.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Tarendai/Shard/issues/102#issuecomment-230098557, or mute the thread https://github.com/notifications/unsubscribe/AADl58rL1SsMPsW6_s0oGNhpY0Hafs2cks5qRlO2gaJpZM4I6xY8 .
We have but 1 issue. If Shards C++ implementation uses a submodule, will that be pulled in via the engines build process, and the submodules inside that. Or will we end up with a Shard that complains at startup that no lua files are present? Will it still pull in the EvoRTS and BAR/BA submodules?
submodules (as Shard itself is) are optional. so the only thing you could do to enforce the presence of a submodule is to add some check into cmake for the presence of a subdirectory.
Do we not already have that? EvoRTS specific logic is in a submodule
My knowledge of CMake is near nonexistent, as is the documentation, and I get the feeling the scope I'm asking this question in is a tiny subset of the scope you're thinking of.
Everybody here is in agreement, we have a plan and would like to execute it. But the spring engines CMake system is an unknown, we do not know if implementing our plan will break Shards Native C++ implementation in the build process or what changes in CMake would be needed and how to implement them On Sat, 2 Jul 2016 at 15:03, abma [email protected] wrote:
We have but 1 issue. If Shards C++ implementation uses a submodule, will that be pulled in via the engines build process, and the submodules inside that. Or will we end up with a Shard that complains at startup that no lua files are present? Will it still pull in the EvoRTS and BAR/BA submodules?
submodules (as Shard itself is) are optional. so the only thing you could do to enforce the presence of a submodule is to add some check into cmake for the presence of a subdirectory.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Tarendai/Shard/issues/102#issuecomment-230103145, or mute the thread https://github.com/notifications/unsubscribe/AADl51W0ETWvFxpAcU5Ttl6iBi8u5IP5ks5qRm-qgaJpZM4I6xY8 .
i don't understand your plan. i only know about "split into submodules" but there are IMHO more reasons against doing that then reasons for doing that.
This isn't a discussion over wether it's a good idea or not, though if you have a better plan please suggest it in a new issue. I don't know how the engines build process will react, will it pull in the submodules? If it doesn't pull in the submodules then it's broken.
Specifically, the plan is to move everything in data/ai/ into a new repository. This way Shards logic is independent from the API implementation.
Right now if you want to use it in a lua AI in your game, you've got all this native C++ code you don't need. If you wanted to use the native C++ AI, there's all this lua AI gadget stuff you don't need. It's messy and complicated, and makes updating things complicated. If you're say the BAR developer, and you want a pure 100% lua version of Shard, you've got to grab this repo, take the stuff out the data/ai folder, delete all the Visual Studio/CMake/C++ stuff, and hem it into the other repo that has the LuaAI gadget loading code. It's a bit of a mess.
The end result of this is that we'd have 3 repos:
- Native C++ AI implementation
- Lua AI implementation using Spring gadget API
- The Shard AI itself ( which itself contains these 3 submodules ):
- BA
- BAR
- Evo
Now AI development can happen in 1 place. Right now there are 2 versions of Shard with duplicated code that cannot be merged together
Ignoring why we might want to do this, or the workflow or processes that might result of it, will this work with the engines CMake build system, or does it require work to be done? Assuming zero knowledge of CMake and the engines build process
will it pull in the submodules
no, it requires a lot of changes to the build scripts. also all build instructions are missing the --recursive parameters.
will this work with the engines CMake build system, or does it require work to be done?
yes, as it must check if the submodules are checked out / exists. else it will fail at some later point / the submodules are just missing.
Where would one begin in order to implement that? Or is there an alternative dependency management system that can be used, similar to composer/bower/npm?
Where would one begin in order to implement that?
https://github.com/Tarendai/Shard/blob/master/CMakeLists.txt#L10
example: https://github.com/spring/spring/blob/develop/tools/CMakeLists.txt#L5
i don't see alternatives to cmake which make sense as they would require a new tool.
i've reverted this in spring's fork:
https://github.com/spring/Shard
submodules cause a lot more problems than they solve.
Problems can be frustrating but we all contribute our own time for free here, being rude is not the answer, and I'd much prefer that you open a constructive discussion on how to move forward when these problems occur.
With that said, I'm disappointed as the defacto project lead of the engine you've barged in here with unfounded and unresearched accusations. It's irresponsible and demoralising, and I don't appreciate you shutting down a discussion on solutions in this thread issue, or derailing it with an unrelated problem that already has a separate issue thread ( #109 ).
Until you learn good conduct befitting of a project lead I've revoked your admin status on here, I see no reason you can't make pull requests to feed back fixes made elsewhere. If you need to vent, GitHub is not the place for it.
What did you revert? None of the changes discussed here were ever made
because i don't want to break it a second time i "removed" the submodles.
Ideally Shard would use a project level dependency manager, such as how PHP has Composer, Javascript NPM, etc etc, letting dependencies be pulled in at build time with success and failure conditions, mirrors, alternatives, etc, rather than as less robust git submodules that need updating.
I'm unaware of any that would do the job here though, perhaps you might know of something that would do the job? Something I can specify in a file what packages, their versions, and locations they can be found
That doesn't work, the C++ AI and the LuaRules AI are different projects with a shared dependency. Game devs aren't going to bundle a Visual Studio project in their games and the native AI doesn't need a LuaRule handler, they also have different requirements about the root of the repository, the current single repo situation is far from KISS
submodules are a mess:
- cloning spring + submodules takes much longer the more submodules it has
- its much more error error-prone: external repositories can disappear
- there is no easy/clean way to remove a git submodule, it requires manual interaction: everybody who compiles spring needs to remove it manually
- the current submodules are not used / maintained (!!!)
- things i forgot
the only clean way to solve your and my issues is to make shard to be able to load the lua scripts from the current loaded game archive.
also for the current situation: remove the submodules and make shard work again or i/we have to remove shard from spring as its not useful when its broken.