CKAN
CKAN copied to clipboard
KSP-CKAN
Define an explicit policy for what ckan mods are allowed to do
@ferram4 (oh yes, I'm poking you in this one) has raised some very reasonable concerns regarding what mods should be allowed to do, and what they shouldn't, if they are to be installable via ckan.
Some mods can execute actions at runtime that will put the GameData folder (or worse) in a state that is inconsistent with the state recorded by ckan. For example, a mod might not only create additional files that wouldn't be removed by the client (see #653), but it might also edit other mods' files as well.
We cannot, obviously, handle every possibility: we can deal with the most common behaviours, but in the end we need to explicitly state what run-time actions are permitted for a mod to be fully compatible with ckan.
Actually I agree with this. I'd further add to it do say that mods should not include dependencies in the mod itself. For instance firespitter, or other 'library' mods that many other mods use. They should list it as a dependency but not include it in the package itself.
This would allow ckan to handle depends with also being able to handle compatibility. For instance mod x works fine with other mods at version 1.2.3, but 1.2.4 might break other mods, which ckan could detect and deal with.
@Sakata-MC bundling other mods is not an issue for CKAN as we just filter those out when installing
"but it might also edit other mods' files as well."
Of course, some "mods of mods" (for example, Environmental Visual Enhancements and Renaissance Compilation) are intended to do this. Though I do not think those edit other mods at runtime, just at installation.
Is there a case where modifying stuff in another mod's folder, including mod config files, is desired behavior?
I'm up for restricting it to a mod based data directory (so things like Active Texture Management don't break), and the gamedata folder the mod installs to.
AFAIK, you should be able to use the module manager stuff to alter parts for other mods, so there shouldn't be a reason why one mod would mess with another mod's file structure, yet this allows for mods to work with other mods data once things are loaded.
Fine, I'll actually step in and say something here, even though I dislike the project.
There are many cases where a mod would want to programmatically modify files in another mod's folder after installation. One example would be an auto-pruning tool for use with procedural parts that renames all meshes and textures associated with non-procedural fuel tanks to prevent those assets from loading and save memory in-game once those tanks are unnecessary. Another example would be modifying mod data that is not stored in ConfigNode format (which MM can't touch) so that it can be loaded outside of the game's normal GameDatabase system in, perhaps, a faster method. In both cases, doing it programmatically is necessary to get around CKAN's no-overwrite policy and to ensure that the mods work on manual installs as well.
In any case, the problem is simply that as of right now, CKAN essentially expects that it will be the only thing to make any changes to files it in GameData, but that's not a valid assumption. As I see it, there are two ways to address this:
1) Add Constraints On Mod Behavior (the likely route)
Declare the mods can only affect files in their own directories. Further, to avoid problems such as #653, declare that they cannot create new files in the directories. Make any other restrictions necessary to ensure that CKAN's state is never made inconsistent.
This policy would require no changes on the code side, only changes on the policy and requirements side: once the requirements are set down, mods would have to have their code audited by someone to ensure that they meet the requirements, lest a mod slip in and start making unauthorized changes to a GameData directory. These audits would probably have to be done when new mods are added and every few releases to ensure that a mod that was previously following the restrictions don't start violating them.
The downsides are that it will inevitably prevent a few mods from being on CKAN ever, some will refuse to be there on principle, and it'll look really sketchy considering it'll involve CKAN setting itself up as a gatekeeper of sorts between modders and users.
It's probably the most likely option though, since it fits quite well with CKAN's current focus on knowing what is installed to a KSP directory, so what's a little more control, right?
2) Care Less About Install State, Start From Clean Slate for Updates or When Uninstalling (the better route)
Instead of focusing on keeping track of the state of all files installed, simply keep track of what was already there when CKAN started installing mods. During mod updates, wipe the entire GameData directory of anything that wasn't there to begin with and reinstall everything on a clean slate.
The main advantage is that suddenly all the mods that previously could cause problems are now supported; every mod is going to be able to handle a clean slate install, so no issues there. Further, it ultimately means less data that CKAN needs to hold on to. In addition, installing from a clean slate means that previous requirements on packages not overwriting files from other packages can be lifted, instead replaced with order-of-operation data if needed. Odds are, it won't be, but it provides for more options. Finally, since CKAN is going to be regularly wiping the slate and rebuilding, it'll be a much more robust system, since any errors will be cleared out often.
The big downsides are that this would require a large amount of what CKAN does in the background to be changed. Further, every installation / uninstallation of mods would result in CKAN doing a lot more work to get things done, but since the mod files can likely be cached elsewhere on the system, it'll only require copying and deleting files, not handling internet stuff.
But the ultimate upside is that it'll allow support for more varied mods with less oversight needed from CKAN itself.
Now, as for the way things are now: you're essentially running a system that works behind the scenes like option 1 but has all the requirements (virtually none) of option 2. That's inevitably going to cause problems when a mod shows up that does something along the lines of what I mentioned in the beginning. It has to collapse into either option 1 or option 2, hopefully sooner rather than later, since all the prerequisites for a support nightmare for modders are right there.
The only problem I see with Option 2 is mods that utilise a config file to keep track of user settings. Blowing away their directories with every upgrade will also wipe out any user changes away from the default settings.
First, is the current policy written down somewhere? I've been tangentially following CKAN's development but I haven't really been involved so I want to make sure I have my facts straight.
Second, option 2 would conflict with how I do config files. I leave the config file out of the release .zip file and create it via code on first run if it isn't present. That way when a user upgrades they don't overwrite the config file and their settings are persistent.
Third (and the reason I asked for current policy above), my suggestion for dealing with this issue would be looking at tracking folders instead of files. Then the policy would simply be to make sure all your mod's files are in the one folder. If you want to modify another mod's files that's fine, just make sure that if you mod's folder is removed, all your files go with it. (Stuff like .craft files and such would still have to work on a per-file basis though.)
Now, I haven't been here for previous development so I'm sure there are good reasons for the current system of tracking things on a per file basis that I'm not aware of, but coming into this fresh at this point my first reaction is to look at tracking just folders, that's pretty much how KSP mods are distributed anyway.
A lot of the beautification mods add config files to existing directories of dependencies. Per-file tracking is required for those. The metadata really needs to track both files and folders, with some tag on the install stanzas to distinguish a folder that should be deleted on a --purge and a folder that is being modified.
Okay, maybe I'm missing something here.
Why do those mods have to put their files in the other mod's directory? -MM Patch: Can be in your own directory. -Referencing a file you call in a .dll? Just point it at your own directory. -Actually modifying another mods files? Well, that mod is going to have to be reinstalled anyway to get it's own settings back when your mod is uninstalled so the other mod's folder will be cleaned up at that time.
Yes, I'm aware mods currently do this but I am not aware of any technical reason they need to do so.
The way I'm looking at this is it is a lot easier for CKAN to change it's policy to work on a per folder basis and make it an official policy change to do so then any of the other solutions I have so far seen suggested. (Note I still have not seen the official policy or the historical reasons for the per-file tracking so I may be missing something.)
Again, exceptions would be required for things like .craft files, those have to go into the default ships directory, but mods themselves should have no problem with this.
In summary, it would boil down to: -A mod can only install .dll files and MM config patches into its own folder when downloaded. -A mod can install (or create) files in other mods directory as long as these files do nothing when the mod is uninstalled. (So no MM patches.) -A mod is allowed to edit another mod's files, but its documentation needs to be clear what it is doing and specify if the other mod needs to be reinstalled when it is uninstalled. (I see no way for CKAN to track this. Yes, it can see a file has changed, but if it sees a changed file in Mod A and is uninstalling Mod B, how does it tell if Mod B or Mod C changed the file in Mod A?)
This way when a mod is uninstalled its directory is deleted. It might leave files behind in other places but they don't actually do anything as the mod is uninstalled.
The reason I'm suggesting this method is because CKAN has no way of actually tracking the GameData folder and following what mod is doing what as it is the KSP executable making those changes. Therefore let's leverage the fact that KSP mods are distributed in their own directories anyway and use that (rather then per-file) tracking.
AFAIK there's no written policy (and if there is one, I should know about it). It would be nice to nail something down, but atm it's mostly case-by-case and the only real hard rules I tend to follow about what mods can/can't do via ckan is that they cannot install an executable (security issue), cannot overwrite another mod's files (currently a tech issue), and cannot install to the game's root folder unless absolutely necessary (a security, consistency, and aesthetics issue).
Those are all just my opinions and personal rules though.
As far as mods that install into other mods folders the best example of that are TextureReplacer-dependent mods.
GameData/BoulderCo/Clouds - Every mod that adds visual layers to planets stores config files in this directory and textures in a subdirectory. I think EnvironmentalVisualEffects is the base dependency for this folder rather than TextureReplacer..
GameData/ContractPacks - All ContractConfigurator contract packs go in this directory, without their own directory.
Note that in both these cases, what is installed are static files with nothing being modified.
I misdoubt me there are more examples; those are just the ones that come to mind.