Fully build release packages in CI

[EwoutH]

Currently (as far as I understand) the release packages are build by hand, following this guide. I would like to propose to implement this process fully automated in CI.

Advantages of packaging fully in CI:

• Builds are fully reproducible
• Release load for maintainers is significantly reduced
• Testing new features, bugs, and dependency updates is far easier, especially on multiple platforms
• We could publish nightly/weekly builds
• It reduces the bus-factor

I personally have experience with GitHub Actions and AppVeyor, so I could help with those two. I tried following the guide once, but got hopelessly stuck. So maybe a joint coding sessions could be a solution for that.

[brandesNW]

Hi Ewout, The instructions in the guide are for folks who want to build NetLogo at home and hack on it. I’m happy to help you out if anything isn’t clear. We do have a CI process which is triggered when someone in our group pushes a branch to our server. Multiple tests are run and the user is notified if any tests have failed. We make use of GitHub Actions for this. We previously made extensive use of Jenkins, but do as much as possible with GitHub Actions. The main file we use for the workflow is NetLogo/.github/workflows/main.yml. Although many of our primitives have multiple tests, we do not have any automated GUI testing. Our CI does not package NetLogo into an executable that is available for testing by internal or external users. That is something that might be of interest to us. We only have a Linux machine dedicated to CI. Packaging and testing on MacOS and Windows are done either natively on the machine of a group member, or using VirtualBox. I am not familiar at all with AppVeyor. What does it enable? Best,

• Aaron -- Aaron Brandes, Software Developer Center for Connected Learning and Computer-Based Modeling

From: Ewout ter Hoeven @.> Date: Friday, December 16, 2022 at 6:33 AM To: NetLogo/NetLogo @.> Cc: Subscribed @.***> Subject: [NetLogo/NetLogo] Fully build release packages in CI (Issue #2071)

Currently (as far as I understand) the release packages are build by hand, following this guidehttps://urldefense.com/v3/__https:/github.com/NetLogo/NetLogo/wiki/Building__;!!Dq0X2DkFhyF93HkjWTBQKhk!Q3xhpv7yEzaIqtAATMZ7Cdx0BUq9ck-Ow3U8pKJuOHQ0jgcb6HRoVcBvott7YMxu0CQ2rzBGbi0fEH_0MziZCtoerBAwpAB2jA$. I would like to propose to implement this process fully automated in CI.

Advantages of packaging fully in CI:

• Builds are fully reproducible
• Release load for maintainers is significantly reduced
• Testing new features, bugs, and dependency updates is far easier, especially on multiple platforms
• We could publish nightly/weekly builds
• It reduces the bus-factor

I personally have experience with GitHub Actions and AppVeyor, so I could help with those two. I tried following the guide once, but got hopelessly stuck. So maybe a joint coding sessions could be a solution for that.

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/NetLogo/NetLogo/issues/2071__;!!Dq0X2DkFhyF93HkjWTBQKhk!Q3xhpv7yEzaIqtAATMZ7Cdx0BUq9ck-Ow3U8pKJuOHQ0jgcb6HRoVcBvott7YMxu0CQ2rzBGbi0fEH_0MziZCtoerBC8KaqXow$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ANSUO2W45IXMQDTOS3F46TDWNRHQ5ANCNFSM6AAAAAATAZXKSU__;!!Dq0X2DkFhyF93HkjWTBQKhk!Q3xhpv7yEzaIqtAATMZ7Cdx0BUq9ck-Ow3U8pKJuOHQ0jgcb6HRoVcBvott7YMxu0CQ2rzBGbi0fEH_0MziZCtoerBC-5_pOhQ$. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[EwoutH]

Thanks for getting back!

Our CI does not package NetLogo into an executable that is available for testing by internal or external users. That is something that might be of interest to us. We only have a Linux machine dedicated to CI. Packaging and testing on MacOS and Windows are done either natively on the machine of a group member, or using VirtualBox.

These two point are indeed exactly what I'm suggesting: package in an executable fully automated, and also on Windows and macOS.

I am not familiar at all with AppVeyor. What does it enable?

AppVeyor is a CI platform mainly aimed at building Windows apps. Because it focused on that, it's sometimes more suited for building those. We can use it for Windows if GitHub Actions proves to be too limited.

Who normally builds the binaries for Windows and/or macOS?

[LaCuneta]

If we're just looking to automate builds of checked-in code for a "nightly" or "weekly" alpha kind of release, I think that's pretty doable without too much effort building on what we already have. Off the top of my head the only big snag in automating the build is that Mathematica's JLink.jar library needs to be available to NetLogo to build the Mathematica-Link package, and that's obviously not generally freely available. Skipping that for nightly releases wouldn't be a big deal.

For platforms, at the moment we run WiX on Windows to build our installer, and we use the macOS tools for bundling, signing and notarizing there. I know there has been progress on the ability to run the latter from another OS. Linux might be package-able from another platform, but it does do some symlinking of included paths, so wherever it is built on will need to support that.

If the goal is to have the actual NetLogo final release packages automatically built and published, that's going to require a lot more work. Just walking through the Releasing wiki page, here are my thoughts on the current manual processes we have:

• Writing release bullets. There is a script that tries to generate this from GitHub issues, but we haven't been quote disciplined or descriptive enough with them to be used without a lot of manual massaging. Maybe the answer is to get stricter about using issues to track the work going into a release, including keeping the title release-notes-worthy.
• Java version bumps. Generally we include the latest of whatever major version at release time, unless a known bug stops us.
  • There is a wiki page to track the included Java versions across NetLogo releases. It'd be nice to auto-update that, too.
• NetLogo Web export page. We have an open GitHub issue to make this auto-fetch the latest from NLW instead of being a static copy.
• Version number and copyright updates. Yep, needs to be automated.
• Release announcement. This is built from the release bullets and the models library changes info. No way to automate this short of some AI thing, as it's just writing out info on the release in plain language.
• Extensions. Yep, needs to be automated.
  • There is the bundled-libraries.conf that needs updating along with this.
  • Sometimes extensions are updated for compatibility with NetLogo changes, and those updates are not released through the extensions manager until the NetLogo release is out. This complicates getting the versions right on those extensions and might be beyond any easy automation.
• Models library. Updating NetLogo's submodule needs to be automated. And note the models library maintains a listing of changes since last release on its wiki that needs to be merged into the release bullets.
• Notarizing the macOS package. Yep, needs to be automated.
• Tagging the release. Yep, needs to be automated, but note that we only want to tag once the release is made official to the public. Until that time, we might find a last-minute bug that necessitates a fix and re-starting the process.
  • It would be nice if the GitHub Release was auto-generated for a final release package (not a beta).
• Publishing the release on our website. Yep, needs to be automated. To be clear, the upload of the page and platform packages is automated already, it's the final steps to make the release visible to the public that are manual at the moment.
• Publish the release artifacts (jars) to CloudSmith. Yep, needs to be automated.

[SethTisue]

Off the top of my head the only big snag in automating the build is that Mathematica's JLink.jar library needs to be available to NetLogo to build the Mathematica-Link package, and that's obviously not generally freely available

(Dealing with that would be a bit of hassle, but doable: make the JAR available somewhere in encrypted form, perhaps just checking it into the repo itself... then add the decryption key to the repo as an encrypted secret.)

[EwoutH]

Thanks for both your replies!

If we're just looking to automate builds of checked-in code for a "nightly" or "weekly" alpha kind of release, I think that's pretty doable without too much effort building on what we already have.

I think most of the benefits are also in the nightly kind-of builds, for rapid testing.

The list of other stuff is really great, we can take a look at the low-hanging fruit there when the nightlies are working.

Seth, the encrypted JAR is a good idea, I think that could work great!

I made another attempt at building on macOS and Windows, and I don't know what changed (probably all the great work around the Java 17 update), but it now worked way better! I opened a PR, https://github.com/NetLogo/NetLogo/pull/2072, which takes the first steps at building on Windows and macOS.

[brandesNW]

Hi Ewout, I have been doing the recent packaging, with some help from Jeremy. The upgrade from Java 8 to Java 17 required changes to our build software because we needed to move from using javapackager to jpackage which required a significant change to the code, and to the structure of our executables. One thing we definitely need to add to our Windows release is a valid code signing certificate. Signing of the MacOs executable is automated, but manual steps are required for the notarization. As I noted earlier there are a lot of details involved in creating beta and final releases. At least a few of them would be needed for daily packaging, as we would want even a non-release NetLogo to show an appropriate version number such as NetLogo 6.3.0-2022-12-19. I use a script together with a template to create a release specific list of the commands and script that need to be run to update the code base, build and package NetLogo and to make it available on our website. I can share this information with you if we go more deeply into the process. Aaron

Aaron Brandes, Software Developer Center for Connected Learning and Computer-Based Modeling

From: Ewout ter Hoeven @.> Date: Monday, December 19, 2022 at 6:56 AM To: NetLogo/NetLogo @.> Cc: Aaron Andre Brandes @.>, Comment @.> Subject: Re: [NetLogo/NetLogo] Fully build release packages in CI (Issue #2071)

Thanks for getting back!

Our CI does not package NetLogo into an executable that is available for testing by internal or external users. That is something that might be of interest to us. We only have a Linux machine dedicated to CI. Packaging and testing on MacOS and Windows are done either natively on the machine of a group member, or using VirtualBox.

These two point are indeed exactly what I'm suggesting: package in an executable fully automated, and also on Windows and macOS.

I am not familiar at all with AppVeyor. What does it enable?

AppVeyor is a CI platform mainly aimed at building Windows apps. Because it focused on that, it's sometimes more suited for building those. We can use it for Windows if GitHub Actions proves to be too limited.

Who normally builds the binaries for Windows and/or macOS?

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/NetLogo/NetLogo/issues/2071*issuecomment-1357546034__;Iw!!Dq0X2DkFhyF93HkjWTBQKhk!VvqjelftGac-x9hWDzN9VgsJnGN21r-gjqM6Fa1lR7AjRqsEQaCeAaislP4OT3izOiYZg6WD_0WdOyzHHy8SHRYHXmGbbRy_zw$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ANSUO2RPNQZVFDFOGEIFRGLWOBEPNANCNFSM6AAAAAATAZXKSU__;!!Dq0X2DkFhyF93HkjWTBQKhk!VvqjelftGac-x9hWDzN9VgsJnGN21r-gjqM6Fa1lR7AjRqsEQaCeAaislP4OT3izOiYZg6WD_0WdOyzHHy8SHRYHXmHNreFIKQ$. You are receiving this because you commented.Message ID: @.***>

[EwoutH]

I had a really nice talk with @LaCuneta last week. On of the things that would help me as a developer tremendously is having automated, release builds in CI. I don't have a full Java and Scala toolchain setup, and to be honest, even if I had, I wouldn't know how to start building NetLogo, let alone it being efficient or best practice.

When working on a new feature, is really nice just to push, grab a cup of coffee, and then have your build be ready and be able to test it instantly on your own system. It would lower the barrier for developers - especially ones which are primarily modellers instead of programmers - significantly, allowing more people to participate in development.

It also has a big impact on the release cadence and ability to quickly ship features. As maintainer of the EMAworkbench and Mesa, I noticed that issuing a (bugfix) release went down from multiple hours to less than 15 minutes once have had a fully documented and almost fully automated release process.

Once you have that, you can ship more often with less effort. Currently there is a period of over a year between NetLogo minor releases. That means once I think of a nice new feature, open a PR and get it reviewed and merged, I still might need to wait for a year before I can use it, or roll it out to students. Having 4 to 6 minor releases means that I can encounter an issue in a course, and have it fixed before the next one starts.

Even more, if you have full release builds in CI, you don't need to wait on official releases anymore. You can just grab a nightly or weekly build. That also means that builds will be better tested and bugs get reported faster, while the developer that developed it still knows the what and why and how and isn't doing something else entirely by now.

Finally, it just saves major headaches and reduces the bus factor of a project.

While I don't know a lot about Scala and Java build systems or SBT, I do have some CI experience and can support with that part.

We don't have to build Rome in one day. To start, we could just start building core NetLogo in CI for one platform (for the tests we already do for Linux). Then we can slowly add components (extensions, the model libary, etc.) until it's a full release build. Once that's up and running, we can do it for the other platforms. And then finally, we could look at all the stuff that's about releases (tag, release notes, etc.) and not release builds.