Formalise Certified Nodes process

[joepavitt]

With our certified nodes program, we have a collection of widgets that customers can trust.

With Marian leaving though, we don't seem to ahve a formally documented process by which new nodes can be added to this list. As such, we should have, documented likely in the Handbook, a clearly defined process by which a set of nodes can be considered "certified"

[ZJvandeWeg]

@joepavitt @knolleary

I think there's a few things here to fix:

1. There's two repos right now, one that generates the verification data, one that publishes the catalogue.json
2. The scanning tool used no longer works -- npm audit might need to be swapped in
3. The bash script generating the output consumes a list of nodes, validates them, and only the ones that passed inspection make it onto the list for next time. The list thus becomes shorter and shorter.

[ZJvandeWeg]

@joepavitt Who started this work?

[joepavitt]

Now? Or for the nodes already done?

For the latter, my understanding was it was all Marian?

[ZJvandeWeg]

Sorry, I meant now. I figured it should've been assigned to someone and I wasn't sure who. The original work I'm aware of :)

[joepavitt]

Sorry, yeah, the "Planning" repo says "Started" as it was started in the past, but the "Development" board entry I've just added shows it's "todo" for someone to take in the immediate future

[ZJvandeWeg]

@joepavitt Just to shortcut the development and for me to hack in my evening I've created https://github.com/FlowFuse/certified-nr-nodes/pull/5/files. I don't think it's production ready, but might shave a few hours of another engineers time. Hope it helps.

It's not my intention to complete this, as my output is unpredictable in delivery time at best.

[knolleary]

To start with, I'm writing down what we have in place today to make sense of how it ties together.

certified-nr-nodes repo

• The certified-nr-nodes repo contains the master list of nodes.
• It includes an audit-modules workflow that runs in a GH action. The action pip installs njsscan and npm installs @sandworm/audit which are then run against each module in the list generating some output. It also rewrites the modules.json file to only include modules that 'passed'.
• As part of the audit script, it appears to do a POST to a FFC hosted NR instance - certified-nodes-prod. I cannot find an instance by that name.
• The GH audit action has to be run manually.

ff-catalogue repo

• The catalogue repo is used to generate the NR Catalogue json used by instances. The repo includes a list.txt file that lists what modules it should include from the public catalog - plus has @flowfuse/node-red-dashboard-2-user-addon hardcoded in from our private registry.
• The gh build action generates the catalogue and pushes the results back to the gh-pages branch - where it gets published under the public url we share.
• There is an audit workflow in that repo that looks to do much the same as the certified-nr-nodes repo - but has never been run.

I'll talk to @hardillb next week as he helped setup the ff-catalogue side of things and may have some context on how the certified node audit was intended to be managed between the two.

---

We need:

1. a single list of certified nodes that is only modified via PR on our part
2. an action that runs regularly against the nodes to verify them.
   • if the verification finds any issues, we need a way to flag that up for manual review
3. the catalogue needs to use the same list - which suggests we do all of this in one repo

I'll review the sandwork/njsscan tools that were previously used. ZJ mentions they no longer work - but not clear how so. npm audit is certainly a good tool these days to monitor for these nodes. The other tools dig deeper into the code and will bring additional value assuming they are made to work properly.

[knolleary]

The certified-nr-nodes is going to be the primary repo for this going forward.

Via https://github.com/FlowFuse/certified-nr-nodes/pull/5, the repo now generates audit reports using data from npm audit and any deprecation notices (which aren't included in npm audit output).

As well as the node catalogue, it generates a one page summary of the nodes to make it easy to check the current state of any of the nodes.

The workflow will run once a day to keep the report up to date.

We will retire the ff-catalogue repo and move the hosting of the catalogue json over to the certified-nr-nodes repo.

The remaining task is to document the process for adding nodes to the list.

[joepavitt]

@knolleary status update please? Think, reading above, it's just the outstanding documentation of the process?

[knolleary]

@joepavitt yes, the status is as written above and as we discussed on Wednesday.

To summarise:

• [x] retire ff-catalogue repo
• [x] switch dns to point catalog.flowfuse.com to the certified nodes repo
• [x] write up documentation on how to add nodes to the list
• [x] investigate how https://flowfuse.com/certified-nodes/ is being generated to update it

[knolleary]

Have completed the tasks with these two PRs to get this overall task finished off for now:

• https://github.com/FlowFuse/certified-nr-nodes/pull/17
• https://github.com/FlowFuse/website/pull/2235

The website part ( https://flowfuse.com/certified-nodes/) is generated by pulling the catalogue.json at build time and generating the content. It caches the content so will only refresh once a day. No immediate changes are needed.