OpenKh icon indicating copy to clipboard operation
OpenKh copied to clipboard

OpenKH ModManager: Add online mod repository browser and downloader with thread-safe image loading

Open Yokimitsuro opened this issue 9 months ago • 12 comments

This commit introduces a new online mod repository browser and downloader feature with several improvements:

  • Adds functionality to browse, view and download mods from online repositories
  • Implements a new UI section for browsing downloadable mods by game
  • Creates a downloadable mods list populated from JSON configuration
  • Implements thread-safe image loading using Dispatcher to prevent UI threading errors
  • Makes URL retrieval methods public and accessible for proper image display
  • Adds proper exception handling during image loading process
  • Updates the available mods list with real and current repositories
  • Optimizes the DownloadableModViewModel constructor to properly initialize properties
  • JSON URL in DownloadableModsService.cs Line 22: private const string DEFAULT_MODS_JSON_URL = "https://raw.githubusercontent.com/OpenKH/mods-manager-feed/main/downloadable-mods.json";

These changes allow users to discover and install mods directly from the application interface without the DependencySource errors that were previously occurring. image image

Yokimitsuro avatar Mar 21 '25 00:03 Yokimitsuro

Only taking a quick look now not a review. but looking at the image this Downloadable Mods section should be its own window not apart of the main view. Some option under Mods that can pull this up in a separate window (can just be a copy of main window but only to show downloadable mods and if they are installed) or a button below the Delete Selected Mod. button. Having this massive window as the new default especially when its not every time you run MM that you want to see downloadable mods.

Prior to an actual review make this its own window (id suggest under the Mods option to open it) that a user can pull up to browse installable mods.

This also would let you use your own variable separate the launch game to only show and install mods into the correct launch game regardless if the main window has that game selected. Or you can not use it entirely since the json knows what game the mods is for you can just show all of them each under its own category and let the user install across all games at once (i can understand only showing mods for a selected game though so a user doesnt miss which game category they are browsing and get confused when it a BBS mod they just downloaded doesnt show up with KH2 launch game selected)

shananas avatar Mar 21 '25 01:03 shananas

Might I make the recommendation to instead of relying on a mod whitelist-type file, we make the mod manager simply pull all Github repositories that have the openkh-mod or openkh-mods tags? Additionally, it was in the talks awhile back to have a "blacklist" in case we ever have any repositories that have stolen or otherwise plagiarized work. I concur with Shananas' recommendation to have a separate window for this, though preferably with a "browse mods" button somewhere on the main window that auto-fetches mods with the aforementioned tags. Ballooning the main window could be an unpleasant change for some people, but also hiding it under a menu so some people would miss it is kinda silly. Maybe just in a corner or at the bottom somewhere, Browse mods from GitHub would be ideal imo. Otherwise, this looks phenomenal.

Vladabdf avatar Mar 21 '25 04:03 Vladabdf

I completely agree with your recommendation. Pulling repositories directly from GitHub using tags like openkh-mod or openkh-mods is definitely a cleaner and more dynamic solution, reducing the need for constant manual updates.

The idea of implementing a blacklist to handle problematic repositories is also excellent. This gives us flexibility and control, helping to maintain quality standards without excessive micromanagement.

Regarding the UI design, I fully concur with both you and @Shananas—a separate, dedicated window accessible via a clearly visible "Browse Mods from GitHub" button would be optimal. We can avoid cluttering the main interface while ensuring the feature remains discoverable and convenient.

Implementation options:

1. Using the GitHub API:

  • Pros:

    • ✅ Clean, structured, officially supported by GitHub.
    • ✅ Less prone to breaking changes.
  • Cons:

    • ❌ GitHub API rate limits requests to 60 per hour for unauthenticated users.
    • ❌ Authenticated users get higher rate limits (5,000 per hour). We would likely need users to provide a GitHub token or implement OAuth to avoid hitting these limits.

2. Scraping GitHub directly (HTML parsing):

  • Pros:

    • ✅ No rate limit constraints.
  • Cons:

    • ❌ Fragile; if GitHub changes their page structure, scraping logic will need frequent maintenance.
    • ❌ Could be slower or potentially violate GitHub's Terms of Service.

Given these trade-offs, I'm inclined toward using the API option to ensure robustness and reliability.

Do you agree with this, or would you prefer we explore scraping further?

Additionally, to categorize and filter mods accurately by game (KH1, KH2, BBS, ReCoM, KH3D, etc.), we'd need to enforce that each mod repository explicitly provides clear categorization, either within the existing mod.yml file or through another standardized method. This would enable accurate filtering when users browse mods.

Does this approach sound good to you? Let me know your thoughts or if you'd like to consider anything else.

Yokimitsuro avatar Mar 21 '25 07:03 Yokimitsuro

My category message was posted to the implementation as is. Since you have a json with categories if we are switching away from that that part of the suggestion may need to be reconsidered. Also iirc we technically already have a blacklist. I forget which file it's in but mod manager does pull from one within a separate openkh repo it just currently has no mods that are blacklisted.

Found the repo here: https://github.com/OpenKH/mods-manager-feed

shananas avatar Mar 21 '25 07:03 shananas

something like this?

Work In Progress image image

Yokimitsuro avatar Mar 21 '25 10:03 Yokimitsuro

I don't like the Mod Manager itself having this functionality for a multitude of reasons, some mods requiring external steps to install for one (such as Re:Fined and Critical Mix). Solid idea, but this would need to be done in an external way.

I am rejecting this PR, if anyone would object they can do so.

In fact, I think it is NECESSARY to have an internal mod search tool. Sometimes, finding a specific mod outside of NexusMods can be tricky, and this could be a great solution to help end users install more mods than ever, in a centralised way, without the need to search all over the internet.

I don't think mods that require external installation steps would be severely affected (like Re:Fined/Critical Mix); it would be as simple as indicating in the respective mod.yml (either through the description or a new boolean field) the need for extra steps to make them work properly.

TheSoraHD avatar Mar 21 '25 10:03 TheSoraHD

I retract all my previous statements. Do whatever.

TopazTK avatar Mar 21 '25 12:03 TopazTK

Im unsure github API is the way to go and half agree with topaz's removed statements. so i have this proposal

  1. keep the json but not in this repo probably move it to the one i linked earlier or the org can make another just for this json. a) it prevents mods that are not a simple click to install (that may be tagged as vald said) from showing up its a good point refined shouldnt show up in this mod browser becuase it will confuse people since the mod does nothing by itself.. b) the json being in a different repo prevents a new release being built every time. also id expect anytime a mod is added the author is aware and ok with it being added ( i dont think this needs to be a question every time but a general hey we have this feature now can i PR to this repo so it shows up in the mod browser if the user hardly uses github or they can PR it themselves)

still havent reviewed the code but no reason to stop working on it itll be merged. from the latest images got to fix the text being the wrong color in dark mode (ik it was a PoC) Then move the json i think adding it to the already exisitng repo for things like this https://github.com/OpenKH/mods-manager-feed would be best. Id be glad to accept a PR with a couple mods into that repo to show its working without the json in this second repo.

shananas avatar Mar 21 '25 16:03 shananas

https://github.com/OpenKH/mods-manager-feed/pull/1

Yokimitsuro avatar Mar 21 '25 22:03 Yokimitsuro

This PR depends on OpenKH/XeEngine.Tools.Public#1

The ModsManager requires the RefreshIcons.xaml resource file that has been submitted as a PR to the XeEngine.Tools.Public submodule. Once that PR is merged, this PR can be updated to use the official repository version.

Currently, the submodule is using a custom branch with these changes to make the application work correctly.

Yokimitsuro avatar Mar 22 '25 00:03 Yokimitsuro

This PR depends on OpenKH/XeEngine.Tools.Public#1

The ModsManager requires the RefreshIcons.xaml resource file that has been submitted as a PR to the XeEngine.Tools.Public submodule. Once that PR is merged, this PR can be updated to use the official repository version.

Currently, the submodule is using a custom branch with these changes to make the application work correctly.

merged

shananas avatar Mar 22 '25 00:03 shananas

updated json

Yokimitsuro avatar Mar 24 '25 00:03 Yokimitsuro

Wanted to check on this. (moving the mod browser to its own separate window ETC)

shananas avatar Jul 02 '25 00:07 shananas

Is this superseded by #1187?

TopazTK avatar Jul 05 '25 13:07 TopazTK

Is this superseded by #1187?

yes

Yokimitsuro avatar Jul 06 '25 14:07 Yokimitsuro