PX4-Autopilot icon indicating copy to clipboard operation
PX4-Autopilot copied to clipboard
verified publisher icon PX4

Firmware builds manifest

Open mrpollo opened this issue 7 months ago • 13 comments

I propose an upgraded workflow for delivering firmware releases that everyone could leverage to download any given PX4 release reliably.

Our current approach is as follows:

  • The PX4 team automates the build of releases and uploads artifacts (*.px4 files) to an S3 bucket in one of three folders.
    1. When creating a new release, build all board targets and variants
    2. Grab all artifacts, including metadata, and upload to an S3 bucket
    3. The S3 bucket has three main directories
    • master: daily builds from the main branch. "master" is the previous name of the main branch.
    • beta: any non-stable release tag, for example: v1.16.0-rc1, v1.16.0-beta1, v1.16.0-alpha1
    • stable: only fully tagged releases, for example: v1.16.0, v1.15.4, v1.15.0
  • QGC has a hardcoded link to the S3 bucket and knows where to pull data for stable, beta, and daily releases/builds.
  • Whenever a user chooses to install a PX4 release, QGC can only offer the latest release for each category. Stable releases, beta releases, or daily ones. (Not taking into account flashing by file)

While this approach is simple, it brings a few caveats that should be addressed.

  • No flexibility, we are only offering a single build per board, even though we build the whole variant matrix for a target, those binaries largely go unused.
  • If we ever decide to split the default builds into per-vehicle types, the current approach won't scale.
    • We are already facing this with the rover builds; users must download binaries themselves.
  • Can only install the latest stable, beta, or daily version. If a user wants to pick a previous version of the firmware (for whatever reason), they need to build it themselves and flash it.
  • No archive of previous release artifacts available.

Keeping track of artifacts

We need to provide a mechanism for the consumers of PX4 build artifacts (binaries) that pairs metadata, like release version, checksum verification for file downloads, and any other relevant information, such as the vehicle type designation.

Work in this area started in 2016 by Don, who was trying to overhaul the firmware upgrade process within QGC. While Ardupilot introduced a manifest, the PX4 project never moved in this direction.

Now that we have moved from Jenkins to GitHub actions for release artifacts, it’s likely a good opportunity to move in this direction, by building on top of the infrastructure we set in place.

My proposal is as follows:

  • Create a manifest that captures all the relevant build variants and their metadata, and host somewhere publicly accessible
  • Consumers of the manifest (such as QGC) would then parse this manifest and be able to get a list of builds available
    • This manifest should be search-friendly, no need to parse for hundreds of variants, when all you need is the latest stable firmware for Pixhawk 6X for a copter.
  • The manifest will contain URL information for where to download the binaries, their metadata, and also what the hash signatures (sha256sum) of any linked files is for validation.
  • Optional: it’s likely a good idea to build a mechanism for displaying relevant information back to the user for a given build, for example “This is an experimental board”, “Upgrade QGC to latest”, in an effort to provide instructions or important information.

Given this is a huge undertaking requiring coordination of multiple stakeholders, I propose breaking up the work in stages.

Stage 1

  • Investigate what the proper schema should be for a manifest
    • Collect all the relevant metadata
    • Propose a file format and hierarchy for the data
    • Work with possible consumers and stakeholders
      • QGC / MAVSDK
      • Discuss possible impact with hardware manufacturers
      • Seek buyin and feedback from PX4 maintainers
  • Update our tooling to support the generation of the manifest (outside of CI)
  • Validate the output of our tooling
  • Decide where and when these manifest(s) should be created and hosted
  • Most likely, append to the upgraded artifact release workflow in GitHub Actions

Stage 2

  • Embed the manifest generation into our CI infrastructure
  • Work with QGC and MAVSDK developers to implement our manifest-based firmware distribution.
  • Write proper tests to guarantee schema compliance.
  • User Testing by developers and the flight test team.

Stage 3

  • Ship the upgraded firmware distribution to a stable release of QGC / MAVSDK
  • User Testing by developers and the flight test team.

Relevant Resources

  • #10676 & #19848 by @junwoo091400
  • https://github.com/mavlink/qgroundcontrol/issues/2797

mrpollo avatar May 06 '25 01:05 mrpollo

What about a custom firmware build server? Ardupilot has a nice tool https://ardupilot.org/planner/docs/common-custom-firmware.html https://github.com/ArduPilot/CustomBuild/

dakejahl avatar May 06 '25 02:05 dakejahl

This issue has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/px4-team-sync-and-community-q-a-may-6-2025/45433/2

DronecodeBot avatar May 06 '25 23:05 DronecodeBot

@dakejahl I don't dislike the custom builder idea; however, I don't know how many people would find this helpful compared to an improved QGC experience.

mrpollo avatar May 08 '25 00:05 mrpollo

@dakejahl I don't dislike the custom builder idea; however, I don't know how many people would find this helpful compared to an improved QGC experience.

It's a nice to have but yeah definitely not a super necessary tool. It's not hard to download the repo and build yourself if you want to customize things.

dakejahl avatar May 08 '25 18:05 dakejahl

Here's a draft of the manifest. Please focus on the items. Let's try to define all of the relevant metadata

[
  {
    "name": "px4_fmu-v6x_rover",
    "manufacturer": "Auterion",
    "hardware": {
      "architecture": "cortex-m7",
      "vendor_id": "0x3185",
      "product_id": "0x0035",
      "chip": "stm32h7",
      "description": "PX4 FMU v6X.x"
    },
    "toolchain": "arm-none-eabi",
    "artifact": "",
    "sha256sum": ""
  },
  {
    "name": "px4_fmu-v6x_bootloader",
    "manufacturer": "Auterion",
    "hardware": {
      "architecture": "cortex-m7",
      "vendor_id": "0x3185",
      "product_id": "0x0035",
      "chip": "stm32h7",
      "description": "PX4 FMU v6X.x"
    },
    "toolchain": "arm-none-eabi",
    "artifact": "",
    "sha256sum": ""
  },
  {
    "name": "px4_fmu-v6x_default",
    "manufacturer": "Auterion",
    "hardware": {
      "architecture": "cortex-m7",
      "vendor_id": "0x3185",
      "product_id": "0x0035",
      "chip": "stm32h7",
      "description": "PX4 FMU v6X.x"
    },
    "toolchain": "arm-none-eabi",
    "artifact": "",
    "sha256sum": ""
  },
  {
    "name": "px4_fmu-v6x_flash-analysis",
    "manufacturer": "Auterion",
    "hardware": {
      "architecture": "cortex-m7",
      "vendor_id": "0x3185",
      "product_id": "0x0035",
      "chip": "stm32h7",
      "description": "PX4 FMU v6X.x"
    },
    "toolchain": "arm-none-eabi",
    "artifact": "",
    "sha256sum": ""
  },
  {
    "name": "px4_fmu-v6x_performance-test",
    "manufacturer": "Auterion",
    "hardware": {
      "architecture": "cortex-m7",
      "vendor_id": "0x3185",
      "product_id": "0x0035",
      "chip": "stm32h7",
      "description": "PX4 FMU v6X.x"
    },
    "toolchain": "arm-none-eabi",
    "artifact": "",
    "sha256sum": ""
  },
  {
    "name": "px4_fmu-v6x_multicopter",
    "manufacturer": "Auterion",
    "hardware": {
      "architecture": "cortex-m7",
      "vendor_id": "0x3185",
      "product_id": "0x0035",
      "chip": "stm32h7",
      "description": "PX4 FMU v6X.x"
    },
    "toolchain": "arm-none-eabi",
    "artifact": "",
    "sha256sum": ""
  },
  {
    "name": "px4_fmu-v6x_zenoh",
    "manufacturer": "Auterion",
    "hardware": {
      "architecture": "cortex-m7",
      "vendor_id": "0x3185",
      "product_id": "0x0035",
      "chip": "stm32h7",
      "description": "PX4 FMU v6X.x"
    },
    "toolchain": "arm-none-eabi",
    "artifact": "",
    "sha256sum": ""
  }
]

mrpollo avatar May 09 '25 19:05 mrpollo

How is each item in the metadata used? Can we define the requirements here for each item in the json?

For example how is the architecture field used? For stm32h7 the cortex-m7 seems redundant since it's implied in stm32h7. Can we automate the construction of some of (all of?) this metadata from the existing information under the boards/target directory? For example parsing nuttx-config/nsh/defconfig for things like: CONFIG_ARCH_CHIP, CONFIG_CDCACM_PRODUCTID, CONFIG_CDCACM_PRODUCTSTR, CONFIG_CDCACM_VENDORID, etc. We might also want to include board_id from firmware.prototype.

dakejahl avatar May 09 '25 21:05 dakejahl

For nuttx targets this seems relatively straight forward but I'm not sure how it would work with linux boards. If we want to automate generating the schema we need to have all of this information available in known locations, ideally without needing to go and add a manifest file to every target and duplicate that information, although if we have to it's not the worst thing in the world.

@katzfey @patrickelectric thoughts?

dakejahl avatar May 09 '25 21:05 dakejahl

Hello! Thanks for the heads up @dakejahl.

We are planning to use the following repository for BlueOS: Px4-firmware-manifest

It was mostly based on the ArduPilot manifest / QGC source code to make it easy to integrate. I did not allocate time to collaborate with QGC yet around the repository. I can move it to the Px4 organization if you prefer and improve it as the official solution for Px4 based on the discussions here. The code is 90% done, we just need to agree with the format.

patrickelectric avatar May 10 '25 18:05 patrickelectric

Hey @patrickelectric, thanks, this is great, there's a lot of overlap in the generator scripts, see mine here, it's a quick and dirty solution I put together as a playground for the data. Taking from the ./Tools/ci/generate_board_targets_json.py script by @bkueng

Before we get distracted by the generation step of the manifest, I would love to focus on the data. We need to ensure the manifest payload represents hardware targets accordingly, considering what the apps that consume this will need. For example, QGC could forego (fallback to) their hardcoded USB vid/pid detection mechanism and fetch the latest list from our manifest instead. Similar for the rest of the metadata that QGC needs from PX4, including Aiframe, and Parameter stuff.

As @dakejahl correctly points out, a lot of this stuff is already stored in config files at the board target level, and for the data that isn't, we could figure out how to include it.

mrpollo avatar May 10 '25 20:05 mrpollo

Things that I believe that are important and it's missing from the proposed (https://github.com/PX4/PX4-Autopilot/issues/24806#issuecomment-2867743837) format:

  • global description: For manufacture / board information
  • board_id: For firmware file validation
  • summary/magic: For firmware file validation
  • bootloader_str: To identify if the board is in bootloader mode, since I believe that not necessary the PID/VID will change

Questions about the proposed:

  • Why manufacture is not in hardware ?
  • I agree that architecture is a bit redundant, BUT, that's an important but, I would believe that is important if it points if a firmware is aarch64, armv7 and etc for linux based boards.

patrickelectric avatar May 11 '25 09:05 patrickelectric

I'm not really sure how this would apply to the Qurt platform builds. We build those at ModalAI and make them available outside of standard PX4 channels. Our download and installation process doesn't involve QGC. I'd say go ahead with your plans and if we ever need to integrate the Qurt platform builds into that process we'll figure it out then.

katzfey avatar May 12 '25 15:05 katzfey

This issue has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/px4-dev-call-may-14-2025-team-sync-and-community-q-a/45516/3

DronecodeBot avatar May 14 '25 14:05 DronecodeBot

This issue has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/px4-dev-call-may-14-2025-team-sync-and-community-q-a/45516/1

DronecodeBot avatar May 14 '25 15:05 DronecodeBot

Manifest PR is here #25414, I'm moving forward with this schema unless there are review comments on the PR.

./Tools/ci/generate_manifest_json.py | jq '.[] | select(.name == "px4_fmu-v6x_default")'
{
  "name": "px4_fmu-v6x_default",
  "description": "Firmware for the PX4FMUv6X board",
  "manufacturer": "Auterion",
  "hardware": {
    "architecture": "arm",
    "vendor_id": "0x3185",
    "product_id": "0x0035",
    "chip": "stm32h7",
    "productstr": "PX4 FMU v6X.x"
  },
  "toolchain": "arm-none-eabi",
  "artifact": "",
  "sha256sum": "",
  "summary": "PX4FMUv6X",
  "image_maxsize": 1966080,
  "board_id": 53
}

mrpollo avatar Aug 12 '25 17:08 mrpollo