butler
butler copied to clipboard
itchio
Mod integration discussion
Looks like it might be time to talk about mods!
One of the only reasons I still plan to release on Steam is because of Steam's powerful mod integration API, which allows players to just hit "subscribe". This feature is pretty critical to me, and it seems to me like it could be integrated into itch.io relatively easily. That'd be great!
To me, there are a few technical details that need to be addressed:
-
Where do the mods install? My game needs access to the mod files, so presumably they would have to install into my game's directory... but these are other devs, acting without my permission. This might be something that needs to be discussed.
-
Is there some way to "register" mods as "for game X", so that people can easily search for mods? Can my game access that list live, and offer links/descriptions? How much control do I, the game dev, have over whether a mod is whitelisted, blacklisted, etc?
-
How do mods react to updates? When my game updates, I need to be able to tag mods as potentially outdated, and when mods update, that needs to be automatically shipped to the client machines. Right?
Those are my big concerns, but I'm sure there are a ton of things to talk about, and I'd love to hear some thoughts!
Couple ideas:
Leveraging Butler
This is more for my own specific game rather than a generalized mod solution, as each game's architecture is different.
Defender's Quest is a highly data-driven game, and so the engine reads nearly all of its information from data files and likewise gets all of its graphics and audio from loose files. Thus mod support is pretty easy -- just provide an alternate data folder with different data.
One thing I would REALLY like is the ability to automatically merge mods (that are designed to be compatible with one another), by diffing Mod A's folder against the base game, diffing Mod B's folder against the base game, reconciling the differences, and producing a Mod C. In the naive case, say Mod A just modifies a single item and Mod B just modifies a single enemy -- these are in two different files, and so it would be easy to grab both diffs and produce a new patch file, provided butler was up to the task.
Then I could create a simple in game interface that lets players toggle mods from a checklist and attempt to use them together (with a notable warning that not all will be compatible).
In this case I would actually use butler as part of my game itself.
Could Itch actually pull off "paid mods" ?
From a strategic perspective, I think Itch.io might be in a position to enable a way to compensate the authors of mods that avoids running afoul of the "paid mods" controversy on Steam.
I wrote an early outline here, but some specifics have changed: http://www.fortressofdoors.com/on-mods-and-markets/
For one, I think Itch's success with helping compensate the authors of free tools (such as Tiled) through the way they frame things is the big difference. One thing that's clear is that a regular ol' donation button doesn't work, because the user rarely sees it, it's tucked away in a corner sheepishly, and it doesn't 'ask for the sale.' Also, you could make it a policy that it's up to the customer what portion should go to the modder and what to the developer and/or itch. (Perhaps the devs could weigh in on what they'd allow for their games).
This would require solving some other more difficult and thorny issues with mods depending on other mods, and as soon as money gets involved then there's the copyright concerns to deal with. But I think Itch could do something interesting here.
In terms of file layouts, the most common approach is to have a "mods/" directory that gets filled with individual mods, such as "mods/AnnasSuperCoolMod/" and "mods/ZekeMakesStuffPretty/". Those directories are typically laid out the same way as the dev's data directory, and when loaded live, the data is typically loaded and merged without actually merging the files.
There are often conflicts, but I think the standard approach is to foist the actual merging and conflict detection off onto the dev, rather than the modding API. After all, every game reacts differently and considers different things to be in conflict.
That said, we need to accept that many mods have dependencies on other mods, or need to be loaded in a specific order. While load order might be something the dev has to handle, it would be super nice if itch.io's mod projects could list other mods as required products.
As to paid mods, I have no opinion on it.
@craigperko what we do in Defender's Quest for mods is to use a system storage directory, usually something like /Username/Documents/DefendersQuest/mods -- it's guaranteed to be writeable, and it almost always has more storage available then something like %appdata% -- does that alleviate some security concerns for you, not having them in the same directory as the app itself?
It probably makes sense for a modding tool to have the concept of enabled and disabled mods, and to be able to disable and re-enable a mod without needing to redo the entire download. (This could be as easy as moving directories around.)
It also might be useful to separate two layers out here:
- Official Itch modding API (for fetching / browsing / downloading / installing mods)
- Using Butler's functionality as part of your game's own internal mod-handling logic
I agree that diffing/merging/conflict-sorting-outing should be on the dev rather than the API, but it'd be super swell not to have to write all my own (crappy) diff/patch tools.
The problem with the user data directory is that I have no way to scan for installed mods. (Edit: shouldn't read tech stuff on a phone, text is too small.)
As for turning mods on and off, many games are putting that in the game's menu UI, rather than willy-nilly loading all installed mods. See Space Engineers, Prison Architect, etc.
On Monday, May 16, 2016, Lars Doucet [email protected] wrote:
It also might be useful to separate two layers out here:
- Official Itch modding API (for fetching / browsing / downloading / installing mods)
- Using Butler's functionality as part of your game's own mod-handling logic
— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/itchio/butler/issues/52#issuecomment-219516851
Support for mods would be great! I'd love to see itch become a positive community for mod makers as well as game makers.
+1 for having conflict resolution done on the game side of things - it seems like proper handling would be very game dependent?
Like @craigperko mentions creating a system for dependency tracking seems to be far more important for making a system where you can say "I want cool Mod X" and having the backend know that you also need other mods for it to work.
Other considerations:
- Version pinning: Often mods can break with a new version of a game - a way to allow players to stay on a specific version would be great (obv. works fine with direct download, does it work in the itch app?)
- Version requirements: Mods could need to be on either a minimum version of a game (e.g. 1.2 or later), or a specific version of a game (e.g. this has only been tested on 1.4)..
- Mod enable/disable: I'm not sure if this should just be game/dev side, seems like maybe yes? Our solution right now is just to have a 'disabled' folder that we move disabled mods into
Looking at the bigger picture, there's also a question of how to best showcase and present mod content. An api to query and display stuff in game would be awesome. On the website maybe collections work to start with?
Lots to think about, but nice to see some discussion around it!
@craigperko You can't? I'm able to scan my game's mod directory just fine for installed mods. Am I misunderstanding something?
@rje:
I agree, having a really robust version untangler system is the heart of this. I for one would recommend that the system strongly embrace the Semantic Versioning standard and perhaps have some way of flagging mods that have departed from that so they can rectify it.
@larsiusprime Ah, I was on a phone, and I misunderstood. Yeah, we're talking about the same thing. I consider that to be a game directory. Sorry for the misunderstanding!
Version pinning is a REALLY complicated topic. I'm not sure anyone's come up with a good approach for it yet, aside from just letting users download obsolete versions of the core game without forcing them to auto update.
Regarding displaying mods in-game: right now, the standard is for each game to design its own mod "spec file" (typically a bit of XML) and have a "thumb.png" or similar for the image. The issue is that this requires the game to download the mod before it can be summarized!
A big opportunity for the mod API here is to have a simple format that we can grab from itch.io's API, like the web widget but super simple (name, link, desc, thumbnail, price, version compatibility, voluntary rating?)
@craigperko Yeah, I think as far as version pinning goes, rather than trying to "solve" this problem, it might be best just to make sure that the tools and freedoms necessary to "make it work" regardless of the circumstances are available. Chiefly:
- Can rollback to any version of the game ever uploaded to Itch
- Can do the same on a per-mod basis
- Mods require SOME kind of version tag & dependency metadata. They can lie about it, be wrong, or misuse it, sure, but setting it as a standard will hopefully help the ecosystem a bit.
- Mods should have their own versioning system and should be able to advertise that they are adhering to the SemVer standard.
Regarding rollback: I don't believe itch.io supports this right now? My files seem to overwrite each other with each upload, unless I missed something.
Also : mods should be able to declare themselves under various licensing schemes, so that we can encourage, say, a CC culture among mods that lets them be easily remixed and used.
Obviously legal probably should weigh in here, because mods inherently use the IP of the original game, but I know in the emulator scene this was sidestepped by distributing romhacks as IPS files.
In my own game, my mod format would allow mods to be described as diffs from the base game set, but obviously not all games are structured that way.
@rje This is not about Butler, but I do have a bit to say:
I STRONGLY recommend going with a metadata approach to loading mods, instead of just having an "active/disabled" directory. This is useful for several reasons:
- You can specify load order.
- You can turn mods on and off per-savegame, rather than for all savegames.
- You can synch mods between users/servers by giving them a list of missing mods or even autodownloading them. <--- multiplayer is a major problem with paid mods, btw
So I recommend having a menu in-game that lists all the mods, and let players turn them on and off, drag them up and down to change load order. Save that metadata and use it to drive mod loading.
<--- multiplayer is a major problem with paid mods, btw
I assume you're talking about paywalled mods specifically? Or would there be some other issue with mods that aggressively encourage-the-user-to-compensate (but are ultimately still free) like they do with Tiled?
In my mind, there's three tiers of mods:
1 - Cosmetics, don't affect gameplay. Minimally invasive. 2 - Mods that add new map or level content, but do not affect the rules of gameplay, possibly including custom cosmetic content. 3 - Mods that affect the rules of gameplay, causing incompatibilities with base content, possibly including one or both of the above.
Each can possibly be outmoded by incompatibilities introduced by new versions of the game, and loading order can be important, but my main focus at the moment for upcoming work is reducing friction as much as possible for custom map/level content. So having the ability, in-game, to browse, search, and retrieve level content seems sort of important to me for this. (possibly allow rating/feedback/uploading? Have a size limitation, so that the client can auto-download files under 10kB or so?)
I come from the line of thinking that Steam's subscription model is more harmful than helpful for browsing level content, as it adds a lot of unnecessary friction for something you just want to click and play.
Anyway, to me this means that user content would require metadata to specify how much of the game they affect, what other mods they depend on having loaded first, as well as what version they are compatible with. Being for an older version doesn't necessarily mean incompatible, but it can be and only the client can really decide on this. For example, for a puzzle game, I can just require including a solution with custom levels, and if the client can't play it back, it knows it is now incompatible and can notify the user/server.
Metadata would have to be enforced on the client side of things, but I don't think it'd hurt to have a list of categories or tags the developer can set as being explicitly recognized by the game.
A category for user translations of games, or listings of what languages a given mod supports, would also be a nice touch, I'd think.
A category for user translations of games, or listings of what languages a given mod supports, would also be a nice touch, I'd think.
This should definitely be its own special category. It would likely be the most common type of mod!
I think allowed mod categories should be specified by the game dev, because it's very much dependent on the game. For example, my system loads mods on the fly depending on when the content is needed. Custom universes or scripted challenges would only be loaded if you specifically wanted to use them right this second, but ship blueprints and ship+crew mods would be loaded on the fly as you explore, as NPC vessels. In turn, custom ship part mods would be loaded if needed - I don't need to load them all right at game launch.
This also makes searching for mods easier, as a lot of the time players are only looking for specific kinds of mods.
I'm going to go over the whole thread again tomorrow & carefully review each point but in the meantime just pitching in on a few important points:
Other considerations:
Version pinning: Often mods can break with a new version of a game - a way to allow players to stay on a specific version would be great (obv. works fine with direct download, does it work in the itch app?)
It's just a matter of implementing it. There's nothing in the itch app that prevents implementing pinning, and it sounds, actually, quite easy.
Regarding rollback: I don't believe itch.io supports this right now? My files overwrite each other with each upload.
If you don't use butler, when uploading+replacing via the web interface, the old file is actually deleted from our servers.
If you do, every intermediate version is kept as an archive. So you can always redownload whatever version (this is not exposed to users or developers, only to admins - but the data is there).
As far as rollbacks are concerned, it's just a matter of generating reverse patches, really (patch that let you go from n+1 to n). They could be much smaller than regular-direction patches (if regular-direction patches added a lot of content) or much larger (if regular-direction patches removed a lot of content), but it's relatively easy for us to generate them in the backend and to add logic so that itch knows what to download and apply.
