ci: check for metadata changes

[mrpollo]

Fixes #24791

• [x] uORB Graphs Script
• [x] uORB Graphs CI
• [x] Failsafe Simulator Script
• [x] Failsafe Simulator CI
• [x] uORB Messages Script
• [x] uORB Messages CI
• [x] Parameter Metadata Script
• [x] Parameter Metadata CI
• [x] Airframe Metadata Script
• [x] Airframe Metadata CI
• [x] Modules Metadata Script
• [x] Modules Metadata CI

uORB Graphs

Our docs need this metadata to generate a pretty graph that shows you the connections between uORB messages and modules.

Example: https://docs.px4.io/main/en/middleware/uorb_graph.html

Generator script

metadata_uorb_graph.sh — generate, compare, and sync uORB graph JSONs
Usage:
      ./Tools/ci/metadata_uorb_graph.sh [--test-only] [--debug]

Options:
      --test-only	Run generation and comparison only; exit 1 if diffs found, without copying files
      --debug		Echo debug info for missing or differing files

Examples:
      # CI mode: fail if docs need updates
      ./scripts/metadata_uorb_graph.sh --test-only

      # Developer mode: regenerate and sync JSONs
      ./scripts/metadata_uorb_graph.sh

Failsafe Simulator

Our docs needs this metadata to generate the web simulator, this allows you to simulate failsafes!

Example: https://docs.px4.io/main/en/config/safety_simulation.html

metadata_failsafe_web.sh — build and sync failsafe webapp metadata files

Usage:
  Tools/ci/metadata_failsafe_web.sh [--test-only] [--debug]

Options:
  --test-only   Run build and comparison; exit 1 if diffs found, without copying files
  --debug       Show full build output and debug info for file comparisons and EMSDK install

uORB Message Reference

Our docs need this metadata to generate reference docs for all uORB messages in PX4.

Example: https://docs.px4.io/main/en/msg_docs/

Generator script

metadata_msg_docs.sh — generate and sync uORB message reference docs

Usage:
  Tools/ci/metadata_msg_docs.sh [--test-only] [--debug]

Options:
  --test-only    Run generation and comparison only; exit 1 if diffs found, without copying files
  --debug        Show full make output and debug info for file comparisons

Examples:
  # CI mode: fail if docs need updates
  Tools/ci/metadata_msg_docs.sh --test-only

  # Developer mode: regenerate and sync docs
  Tools/ci/metadata_msg_docs.sh

Parameter Metadata

Our docs need this metadata to generate a comprehensive parameter reference.

Example: https://docs.px4.io/main/en/advanced_config/parameter_reference.html

Generator script

metadata_parameters.sh — generate and sync PX4 parameter reference documentation

Usage:
  Tools/ci/metadata_parameters.sh [--test-only] [--debug]

Options:
  --test-only    Run make target and comparison; exit 1 if diffs found, without copying file
  --debug        Show full make output and debug info for comparison

Aiframe Metadata

Our docs need this metadata to list and describe all supported airframes.

Example: https://docs.px4.io/main/en/airframes/airframe_reference.html

Generator script

metadata_airframe.sh — generate and sync PX4 airframe reference documentation

Usage:
  Tools/ci/metadata_airframe.sh [--test-only] [--debug]

Options:
  --test-only    Run make target and comparison; exit 1 if diffs found, without copying file
  --debug        Show full make output and debug info for comparison

Module Metadata

Our docs need this metadata to provide up-to-date reference for all PX4 modules.

Example: https://docs.px4.io/main/en/modules/modules_main

Generator script

metadata_modules.sh — generate and sync PX4 module reference documentation

Usage:
  Tools/ci/metadata_modules.sh [--test-only] [--debug]

Options:
  --test-only    Run make target and comparison; exit 1 if diffs found, without copying files
  --debug        Show full make output and debug info for file comparisons

[hamishwillee]

1. The flaws in https://github.com/PX4/PX4-Autopilot/pull/24980#issuecomment-2960578032 can be ignored. New messages must be added manually to the sidebar (and laways have been) - so you'd fix that in your own build but not this PR
2. The ROS integration test errors are showing up everywhere. They appear to be related to missing gazebo libraries. I don't see how they could be related to this.

I'll give the actual commands a test next.

[hamishwillee]

This particular build succeeded for all the builds except failsafe-web - which fails due to your "tmake" component missing dependencies. It all works locally.

I tried this locally and get the same results. Note though that this is not great test, because I already have emscripten SDK installed AND this branch is up to date with metadata - so we don't every actually get to fail.

Upshot "It seems good", modulo that failsafe web test.

[hamishwillee]

Last of all, note that (as previously discussed), this PR creates CI that is run on each PR. A developer creating a PR is expected to rebase and fix up any metadata errors immediately before merging. I still think better to run on merging into main, but whatever.

This won't do metadata deployment outside of docs scope - such as sending params to QGC.

[github-actions[bot]]

/en/msg_docs/ArmingCheckReplyV0.md

• PageNotInTOC: Page not in Table of Contents (/home/runner/work/PX4-Autopilot/PX4-Autopilot/docs/en/SUMMARY.md)

/en/msg_docs/EventV0.md

• PageNotInTOC: Page not in Table of Contents (/home/runner/work/PX4-Autopilot/PX4-Autopilot/docs/en/SUMMARY.md)

[mrpollo]

The last remaining issue is the Failsafe Web metadata, which seems to be generating different data locally on my development environment vs CI environment under Ubuntu 24.04. I even ran the script under the same docker image that CI is using and I have the same error. I think we gotta look into this before we move forward.

Here's a diff between the CI generated file and my local environment generated file

https://gist.github.com/mrpollo/efead1d01c419567e4645930d9a36b34

[hamishwillee]

The last remaining issue is the Failsafe Web metadata, which seems to be generating different data locally on my development environment vs CI environment under Ubuntu 24.04. I even ran the script under the same docker image that CI is using and I have the same error. I think we gotta look into this before we move forward.

So are you saying that building locally you get the same output as when using the CI docker image (locally), but that produces a different output that running in the github CI environment?

1. Dumb question, but when you run this in the docker image the sources and environment are entirely in the docker image right. i.e. you're not running on sources on your local machine?

2. What I'm making sure of is that the emscipten versions used in each case are the same PX4-Autopilot/_emscripten_sdk/node/20.18.0_64bit/bin/node

[hamishwillee]

I am fairly certain this is the result of using a different emscripten or node version. I ran two builds, first with the _emscripted_sdk that was installed the very first time you create a PR that I had to test. Then I deleted that and ran the build again. The results for index.js are similar to what you see:

IN particular I can see that my node updated quite a lot /_emscripten_sdk/node/22.16.0_64bit/bin/node

I suspect your local build is also out of date. I don't know though how the docker could work unless it builds against the local source including the local emscripten SDK and node

[hamishwillee]

@mrpollo Can we please move this forward.

[hamishwillee]

Every day waiting for this this makes my life suck a bit more.

[github-actions[bot]]

This pull request has been closed after being marked as stale with no further activity. Thank you for the time and effort you put into this contribution. If you’d like to continue the discussion or update the work, please feel free to reopen it or submit a new PR.

[dakejahl]

I don't want @hamishwillee 's life to suck. What's the status on this?

[hamishwillee]

I don't want @hamishwillee 's life to suck. What's the status on this?

My life still sux (in this respect). Still, at least v1.16 builds (at all) so that's something.

@mrpollo Any kind of ETA on this?

If you're not going to work on it, shall I? These can run in a local PR but they need to run on submission to main, ideally as some kind of pre-merge hook.

[mrpollo]

Now that things have settled down, I think the best way forward is to automate this tools so they run post-PR just like we had before, so i will be making that change, and updating this branch.

I'll keep ya posted.

[hamishwillee]

Thank you. Note that I am completely happy with them being able to run locally too - makes testing easier. Just they need to run on injection into main to guarantee accuracy and avoid issues related to different toolchains on machines.