[Enhancement]: Refactor our system for GAME FEATURE tracking (not DEVELOPMENT TASK tracking)

[jonbarrow]

Checked Existing

• [x] I have checked the repository for duplicate issues.

What enhancement would you like to see?

GitHub Projects are cool and powerful, but they are very annoying to work with and have several issues. GitHub has now moved to ProjectsV2, which is not available on the older REST api. This means we are forced to use their GraphQL api. While GraphQL is a powerful tool, it too is annoying to work with most of the time due to the way queries are structured

GitHub in particular has a bad GraphQL api in my opinion. Since every item in a ProjectV2 is considered an "issue" internally, we sometimes have to use some jank/heuristics to actually get the data we want. See https://github.com/PretendoNetwork/website/blob/0a051d1cde2d9cd51e5a4b408f6ac32a75c22f0e/src/cache.js for how we currently handle this

The GitHub GraphQL api also has historically been unstable. In our case, each ProjectV2 is linked to a games repository. Whenever a repository is renamed (like in the cases of renaming the split game-authentication and game-secure repositories into just a single game repository), this completely breaks the link between the ProjectV2 and the repository in the GraphQL api. In the GitHub interface things function as normal, but when querying for the ProjectV2 via GraphQL GitHub throws cryptic errors. The only solution to fixing this is to completely delete the ProjectV2 and remake it from scratch

Because of these issues, we should consider options for moving away from GitHub Projects tracking feature progress in games. Other forms of tracking, like bug reports and such, can still be a ProjectV2 on GitHub

Any other details to share? (OPTIONAL)

Some immediate options off the top of my head (feel free to suggest more), and their concerns:

Roll our own

Pros:

• Full control, so we can tailor everything to our exact needs

Cons:

• Rolling Our Own:tm:, and all the issues that implies (you know the gist)

Trello

Pros:

• Very well known/trusted. Basically the defacto task tracker
• Very easy to use
• Is a kanban board like we want

Cons:

• We exceed the free tier limits (only 10 boards, and only 10 users), and the paid tiers are expensive for a team our size. $5/m per person at the cheapest, which would cost more than some of our old servers to run. Even at just 10 people, that's $50/m for just a kanban board. Also prices are "per month" but billed yearly ($600 all at once for 10 people), if we did monthly it's $6/m per person. This isn't TERRIBLE in terms of price alone, but it feels like a lot for JUST a kanban board
• Kind of annoying to use the api. Traditional api keys are no longer supported. They require you to create a "Power-UP" which requires providing Trello things like multiple contact methods, providing an "iframe connector", etc. Then you have to generate an api key, and use this to "connect" with Trello and get an "api token". Only once you have an api key AND token can you actually use the api, and getting the token can ONLY be done via manual interaction with the browser

Azure DevOps Boards

Pros:

• We already use Azure for things like secret key stores, so this fits nicely into our workflow
• Once you're at the board (which will be touched on in the cons) it feels basically just like Trello

Cons:

• It's Azure, which means dealing with the Azure UI. It takes multiple steps to even find the boards, since you have to travel to them by first going through DevOps Organizations
• Paying for licenses per user, plus having to set users up which is kind of a pain for just a kanban board in the end
• The DevOps Boards api mentions OAuth2, so it seems like traditional api keys are also not supported

[jonbarrow]

To clarify, this is about user-facing tasks. Not development tasks. For example this would be about tracking features such as "Add 100 Mario" for SMM, which under the hood might be several development tasks. Development tasks should still always be tracked using the issue/task tracker here on GitHub. That's how it currently is setup, and that's how it should stay

I'm going to copy-paste some feedback I got on this issue on Discord for transparency reasons:

@CaramelKat said:

I’ll try to respond to this a little later but re: Azure Dev Ops, I use it daily at work and I have opinions™️ about it

@binaryoverload said:

Azure DevOps is a PITA, do not touch with a 6 foot barge pole

It makes GitHub look like the golden child in comparison My honest opinion on the project tracker: JSON file

It doesn't change that often to need to have a CMS-like system, and will avoid any development headaches with APIs and changes in the future Or!

Just use a special GitHub project with custom fields and don't link them to repos

Often the "features" we want to display don't map to actual development features (usually it's a one feature to many tasks or the other way round) so having custom tasks that can be easily renaming and shifted about without affecting the development side would work well I think

So it seems like Azure DevOps is a strict NO-GO

I responded to @binaryoverload with the following:

The fact that “features” don’t often map to actual development features is the main reason why I reject feature request issues people make for specific in-game features. Like if someone were to open a “add 100 Mario” feature request to the SMM server, I would reject and close the issue under the premise that. All Nintendo Network features are planned, and creating “feature request” issues for them does not change when that happens. The issue tracker should be used for actual issues or requesting new features

So that does already basically line up with what we’ve been doing

To which @binaryoverload followed up with:

Even so, you can have development features like "Implement new version of nex lib to support matchmaking" and "Add Postgres support for matchmaking" but you want the feature on the website to be "Matchmaking"

In these cases it's best to split the development features and the user features, otherwise you end up: a) confusing users because they don't understand developing features b) confusing developers because you have to make issues with the constraint that they need to have understandable titles for the public, which more often than not will not reflect the actual issue that needs to be done

To which I said, to get on the same page:

Yeah that’s essentially what we’ve already been doing. Issues on repos are not what is shown on the website, the website uses separate draft issues on Projects that are just linked to the repos. The draft issues display what the users would likely actually care about (IE, “Add 100 Mario”) but on the development side of things that might be like 5 actual tasks

This gives us 2 more options to consider:

JSON files in the website repo

Pros:

• No need for a CMS or APIs at all, completely offline/local
• Tracked with a git history so we can clearly see changes
• Simplest to set up

Cons:

• Is a blocker for wanting to update new features. Now instead of just updating a board and having the website automatically update itself, there needs to be an actual commit/PR to update the JSON. This would slow down the rate at which tasks are updated on the website, and means the website has to be restarted/redeployed just for these minor changes. This can especially be annoying if we have multiple people looking at multiple games and trying to add/work on features
• Makes it more annoying for the dev team to see the tasks. Using JSON files means that we either have to look at the JSON files themselves to see what state a task/feature is in, which can be annoying to read, or check the website itself, which is also somewhat annoying given how the website displays things (it's aimed at end-users, not us)
• Would clutter the git history with arguably unnecessary commits
• Many task trackers like Azure, Trello, etc. let you add notes/context/whatever to tasks, which we would lose with this unless we add the notes to the JSON files and either hide them on the website (forcing us to read the JSON for them) or add them onto the website (and is that really necessary?)
• We run into synchronization issues. Someone might have notes on a task, but only in their local copy. We don't see other people's works until they decide to push (which has sometimes been an issue for us in the past)

Continue to use GitHub Projects, but do not link to repos

Pros:

• Don't really need to change the setup that much
• Still keeps things centralized on GitHub (game feature tracking remaining close to development task tracking)

Cons:

• Everything I listed about GitHub ProjectsV2 being annoying
• If not linked to a project, we lose the ability to add a link to the repo on the website (though in fairness, that same would happen for other tools as well unless we rolled our own)
• Everything is an "issue" internally. Which means that each "card" is really just a draft issue, and we can't bring it out of draft unless the issue is tied to a repo. I guess we could make a "feature-tracker" repo or something for these, but that's kinda hacky/messy? (the ProjectV2 would not be linked to a repo, but the issues themselves would be)

[jonbarrow]

I also went ahead and changed the title/scope to now include staying with GH as an option, and approved the request. Since even if we do stay with GH, we need to change how that works because our current setup is just busted in some areas