How feasible is to use CKAN for other games?

[speeder]

When I used to play KSP (it was a long time ago), I enjoyed CKAN a lot.

Right now I am trying to mod the game Stardew Valley, but it became obvious that some serious way to handle conflicts and dependancies is needed.

So I thought of 3 potential options:

1. writing a CKAN clone from scratch

2. forking from CKAN

3. this idea came after I talked with spacedock people on IRC, they are rewriting spacedock to support multiple games, thus, why not make CKAN support multiple games too?

So, what you people think about the second or third option?

[politas]

For reference, this is the official forum site where mods are discussed: http://community.playstarbound.com/forums/stardew-valley.72/

[mheguy]

@pjf's vision for CKAN from the start was as a tool that could easily be adapted for other games or other uses. I'm confident saying that he'd love to see a fork adapted for another use.

[pjf]

@speeder : As @plague006 mentions, the CKAN is MIT licensed exactly so folks can use it for other games and purposes. Right now we've got KSP coded in as a fixed class, but I would love to see this replaced with some sort of IGame interface which allows multiple games to be supported. That can be as part of a fork, or as part of our mainline client.

I need to play catch-up, but we've got a PR to add Curse support ( #1608 from memory ), and I have some catch-up to play with Nexus as they've also expressed interest. Both are important if we're expanding which games we're supporting.

So yes. Please use the CKAN for anything you think is appropriate; if we've got licenses or other factors standing in your way, we're doing something wrong.

[dbent]

I would support an effort to extract out any KSP specific bits into an IGame interface (or series of similar interfaces IGameComparator etc.). In an ideal world the CKAN client would just contain the core package management functionality and then different games would basically be implemented as plugins so a single CKAN install could handle multiple games.

The spec would also have to be updated to be KSP-agnostic:

• ksp_version -> game_version
• Install targets could no longer be an enumerated list of paths (GameData, Ships/SPH, Ships/VAB, etc.)

And mayhaps some other stuff I can't think of right now.

[Dazpoet]

I would be very much in favour of an effort in this direction but I think it would require some bungling with the repository structure. Maybe the core project should be the client and named something like CxAN for just the Comprehensive Archive Network which would then have the plugins mentioned by @dbent above as seperate repositories. Metadata would also have to be divided into gamespecific repositories I'd assume.

Another kerbal'ish reference which should perhaps be changed is that kref could be changed to gref, assuming the k is due to kerbal-something?

Just a scribble of potential structure (I'm just plugging some games in to show) might be:

-- CxAN-Client (current CKAN repository -CKAN plugin code)
-- CKAN-plugin (Kerbal Space Program)
-- CSVAN-plugin (Stardew Valley)
-- CSAN-plugin (Skyrim)

-- NetKAN (KSP)
-- NetSVAN (SV)
-- NetSAN (Skyrim)

-- CKAN-Meta (KSP)
-- CSVAN-Meta (SV)
-- CSAN-Meta (Skyrim)

If the project grows I'd assume that some roles and groups might need to come into existance in a more strict way than currently so say something like this:

-- Owners (Can do whatever they want and are awesome, you petition these guys to create your new and awesome plugin repositories should you want them under the main repository flag)

-- Core-programmers (Can merge things in CxAN which would include the NetCAN and are responsible for spec-updates. Can also merge in each plugin and will help new plugindevs get started)

-- X-Game-Plugin-Programmer (Can merge things in specific games plugin)

-- Metadata-Maintainers (Can merge things in each NetXAN and CxAN-Meta and will help new communities get their metadata started)

-- X-Game-Metadata-Maintainer (Can merge things in X-game specific metadata archives)

Of course nothing would prevent a single user from belonging in multiple groups or a game community from forking and just having their own repository for their metadata and plugin. Specifically though I'd assume that some people might want to start in e.g. X-Game-Plugin-programmers or X-Game-Metadata-Maintainer to get their feet wet and then move up the hierarchy, much like many of us perhaps not so code-y people have done when we started scrubbing around in NetKAN and CKAN-meta :)

Now I have null experience with things like these in this line of work so I would love to be told why I'm totally wrong/right/far-out :)

[politas]

Daz's outline seems sensible to me.

[netkan-bot]

Hey there! I'm a fun-loving automated bot who's responsible for making sure old support tickets get closed out. As we haven't seen any activity on this ticket for a while, we're hoping the problem has been resolved and I'm closing out the ticket automaically. If I'm doing this in error, please add a comment to this ticket to let us know, and we'll re-open it!

[pjf]

Totes opening this, since I'm pretty sure we want to kep it around. Thanks for trying to be tidy, @netkan-bot . :)

[Dazpoet]

I'm slapping some important loocking, and hopefully correct, labels onto this.

[speeder]

Thanks everyone!

I don't came back yet to this due to me being busy trying to earn some emergency money to save my computers (I have a MacMini and a laptop, and both are trying to die on me... the MacMini yesterday started to hit 98 degrees C every time I hit "compile" on XCode :( )

Still, back on topic, I liked Dazpoet proposal, it is much better than anything I could imagine :)

Just by the way, people tend to call Stardew Valley "SDV", I don't remember why now, I think there is another game called "SV" popularly. (or because SV is Sillicon Valley).

[trakos]

Hey guys,

I'm maintaining a CKAN fork for Factorio: https://github.com/trakos/CFAN (very creative name, I know).

Currently it's a bit simple, with a lot of variables still called KSP and without tests, but it works. I'm thinking of rewriting it in the more generic way so that other games could use it. Would you be interested in merging that if I would manage to do it?

[politas]

I'm sure @speeder would be happy to help with that, and this issue makes it pretty clear that making CKAN into a multi-game mod manager is definitely a popular idea.

[techman83]

@trakos Making CKAN -> CxAN was something we identified pretty early on. I'd love to see across multiple games.

[dbent]

@trakos

I'm thinking of rewriting it in the more generic way so that other games could use it.

Cool.

Would you be interested in merging that if I would manage to do it?

Sure, but I strongly advise against trying to do it as one mega-merge, you'll inevitably get out of sync and the merge conflicts will be a pain in the butt to resolve leading to frustrations all around.

Instead, I recommend isolating small, individual KSP bits and merging those one at a time. You reduce the risk of drift and merge conflicts and we can review and advise you on your code as you go along rather than having to deal with one huge merge.

[Dazpoet]

A dangerous, for some of us, idea might be to create CxAN-Meta and NetXAN against spacedock for the games they support the same way we already do with KSP mods? That way when we sometime down the line add support we'll already have some of the architecture in place? I don't think the workload will be immense since I guess spacedock won't be the goto place for mods for the other games for a while.

The initial thing we'd have to figure out is probably the install stanzas and Jenkins testing (which won't work since we lack clients) but apart from that we could just start catching things from a bot. Even if we don't merge them we'd start gathering a comprehensive list which would prevent us from having to vaccum spacedock for mods later?

EDIT: Thinking about it I guess there's quite a few issues with gathering metadata already since we don't have a proper schema nor tools but... I still think it might be a good place to start?

[politas]

Not a lot of point in collecting data without a schema, but setting up some infrastructure can't hurt.

[techman83]

The infrastructure needs some attention. The testing infrastructure not so much, it's fairly quiet and can spin up containers on demand (we could expand out into AWS's container service as well). NetKAN as it sits it probably wouldn't scale, however I have long had ideas about how to fix that. Inflating with NetKAN is CPU intensive, but it'd be trivial to use some http/api checks to check things more frequently and run a full index daily (to catch edge cases).

[trakos]

Sorry guys, but after reading your ideas I don't think I'm the right person to do it. My idea was quite different.

I thought of slowly moving some parts of CKAN to a separate CxAN repo that could be used by other projects. And I thought CxAN wouldn't contain anything netkan-related or build-related.

And, if I understand correctly, you just want to include support for other games to CKAN and Netkan basically?

[politas]

@trakos , are you intending to manually keep all mod versions up to date? That's going to get pretty tedious for you.

[trakos]

No, I still have netfan (fetches mods mostly from factoriomods.com), it's just small and written from scratch. Adapting and launching entire mirrorkan would be a lot of work for me.

[politas]

Perhaps your netfan code may be usefully combined with some Spacedock-scraping code in CKAN?

[trakos]

Factorio modding support is kind of in the void, because game dev claim there will be official mod portal in next major version. It will probably mean end of current factoriomods.com, so I'm not sure if it's worth adding support for it here.

Anyway, why do you ask, do you still like the idea of trying to backport CFAN?

[politas]

I like to explore the possibilities and make sure people aren't making decisions based on faulty assumptions. My C# skills are very limited, so I'll not likely be doing much if any work on it, myself. I like the idea of CKAN growing into a more generic tool used for multiple games, since the larger a project is, the more eyes there are on the code.

[HebaruSan]

Multi-game support has been added to CKAN for KSP2. There's an IGame interface that defines everything the client needs to know about the game, and the back-end has been significantly re-architected to handle multiple games with different repositories, etc.