Reduce load on macosx-aarch64 machine in RelEng JIPP

[HannesWell]

In the RelEng Jenkins instance we have two build machines running on Mac OS X:

• b9h15-macos11-x86_64
  • macOS 11 (Big Sur), no login session, hosted on MacStadium
• nc1ht-macos11-arm64
  • macOS Ventura 13.0.1

According to the name and description one has a x86_64 and one has a aarch64 CPU.

At the same time we have two I/Y-build test configurations for Mac OS:

• macosx-aarch64-java21
• macosx-x86_64-java21

But for a reason unknown to me both run on the the macosx-aarch64 machine. As the full test suite runs between 2h and ~4.5h having two of them per I/Y-build imposes a heavy load on that one machine and blocks it for quite some time. Especially if extra builds are triggered out of schedule and in times where Y-builds run daily (close before Java releases). This is especially problematic as the SWT and Equinox native binaries are build on these machines too, but these builds are blocked (and often time out) when I/Y-build tests are running. To mitigate the latter other suggestions exist, like

• https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/4231

(Although I'm not sure if it's worth the effort given, that we hopefully 'soon' can use the FFM API in SWT).

But regardless of all site-notes, I think we should reduce the number of test configurations executed on the macosx-aarch64 machine.

Ideally the macosx-x86_64-java21 configuration would run again on the macosx-aarch64 machine. But as it probably was like that in the past I wonder why it changed and if the macosx-x86_64 machine can be fixed to make it run again?

And AFAIK at least in the mobile area all new MacBook laptops use only Apple's ARM based CPUs? I don't know how the situation is in the desktop area but I wonder if x86_64 is still relevant and used for Mac? Is this an architecture just used on legacy Mac systems? If the latter is correct, should we just drop the macosx-x86_64 test configuration? And could we even drop support for macosx-x86_64 in Eclipse entirely?

[HannesWell]

And AFAIK at least in the mobile area all new MacBook laptops use only Apple's ARM based CPUs? I don't know how the situation is in the desktop area but I wonder if x86_64 is still relevant and used for Mac? Is this an architecture just used on legacy Mac systems? If the latter is correct, should we just drop the macosx-x86_64 test configuration? And could we even drop support for macosx-x86_64 in Eclipse entirely?

A quick web-search showed that all new Mac laptops and desktops are ARM based and that the upcoming macOS Tahoe (to be released next week) is planned to be the last macOS that will support x86_64: https://en.wikipedia.org/wiki/Mac_transition_to_Apple_silicon#2025

So it's not yet time to fade out support for macosx-x86_64 in Eclipse, but that time will probably come, depending on how long Apple will support macOS Tahoe. According to https://www.zdnet.com/article/your-old-macbooks-days-are-numbered-as-apple-confirms-end-of-support/ this is in about three (or five) years:

Intel-based Macs will continue to receive important security patches when macOS Tahoe rolls out later this year.
The old Mac model running Tahoe will receive updates for the next three years.
This includes updates available as part of macOS 26 and an extra two years of security patches.

For that time I think it would be good to keep I-build tests for macosx-x86_64, but for Y-builds I wonder if two mac configurations are relevant (Y-builds are also not tested on Windows)? https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/blob/b2245e2a0cddfdf41ee144a78442262c049dab38/JenkinsJobs/Builds/build.jenkinsfile#L13-L14 @stephan-herrmann can you help us here?

[stephan-herrmann]

@stephan-herrmann can you help us here?

I personally am not very keen on testing Y-builds on several platforms with "old" java versions. The most interesting test is always the one on current Java EA. So, rather than testing different mac platforms, adding windows and ideally with the current Java EA would make sense to me.

I actually wonder, why this is disabled: https://ci.eclipse.org/releng/job/YPBuilds/job/ep437Y-unit-win32-x86_64-java21/

[HannesWell]

The most interesting test is always the one on current Java EA. So, rather than testing different mac platforms, adding windows and ideally with the current Java EA would make sense to me.

That should definitively doable. I'll have a look at it (probably not immediately, but hopefully not too far away).

I actually wonder, why this is disabled: https://ci.eclipse.org/releng/job/YPBuilds/job/ep437Y-unit-win32-x86_64-java21/

I asked myself the same and maybe even asked you or somebody else from JDT, but didn't get a definitive answer. I can also set up Y-build tests for Windows with the latest EA java version.

[HannesWell]

@Phillipus, @BeckerWdf, @soethoff as far as I know you are Mac users. I'm trying to make the x86_64 Mac maschine work again and therefore asked the IT team to upgrade the OS to the latest one available:

• https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/6621

Can you tell if this would be a problem for the compilation of native binaries in SWT or Equinox? E.g. would it make these latest version a hard requirement to use SWT or the Equinox native launcher?

[Phillipus]

@HannesWell The version of macOS that the native binaries are compiled on affects SWT.

If you remember this from https://github.com/eclipse-platform/eclipse.platform.swt/issues/1012 where we found that running from Eclipse spawned the child Eclipse to run on Java that was compiled on a later version of macOS.

I opened https://github.com/eclipse-equinox/equinox/issues/495 as a head up. Whether that is still relevant I don't know.

[HannesWell]

If you remember this from eclipse-platform/eclipse.platform.swt#1012 where we found that running from Eclipse spawned the child Eclipse to run on Java that was compiled on a later version of macOS.

I vaguely remembered that there has been problems in the past, but didn't recall the exact issues. So thank you for providing these pointers.

From https://github.com/eclipse-equinox/equinox/issues/495#issuecomment-1938561678:

Is it likely that the Mac build process will one day be updated to macOS SDK 14 (Sonoma)?

I wonder if there's anyone who could answer this question, or direct me to the right place to ask it?

I hope that we can use this issue to answer that question.

According to https://eclipse.dev/eclipse/development/plans/eclipse_project_plan_4_37.xml#target_environments the compatibility matrix for Apple macOS says that version 13, 14 and 15 are supported.

According to this macOS version history 15 is in fact the latest version, but the successor 26 (large step 🤔) is to be released next week. And I was told that Eclipse actually intends to just supports the latest two versions of Mac OS (@akurtakov is that correct?). So from next week on, this would be just 15 and 26.

So should we ask the infra team to upgrade both machines to 15 (if that's still possible with x86_64 as Apple fades out support for it as discussed above)? Or are there many users on these older versions of Mac OS? At least above's overview indicates that 13 is still receiving security updates, but I don't know until when.

If we don't want to change the support matrix, we should still update the macosx-x86_64 machine to Mac OS 13, shouldn't we? I don't know if we have a benefit from updating to a later versions? My main goal is to make the macosx-x86_64 machine usable (again) as long as we test on macosx-x86_64. This of course doesn't require to update the macosx-x86_64 machine.

[HannesWell]

If you remember this from eclipse-platform/eclipse.platform.swt#1012 where we found that running from Eclipse spawned the child Eclipse to run on Java that was compiled on a later version of macOS.

Btw. is it possible, e.g. through some compiler flags, to target older versions of the Mac SDK on newer OS version? Because while one wants to compile low, we want to test on the latest versions as well.

[Phillipus]

And I was told that Eclipse actually intends to just supports the latest two versions of Mac OS (@akurtakov is that correct?). So from next week on, this would be just 15 and 26.

Hmmm.... There's a lot of Mac users who are still on 13 and 14 which are still supported by Apple. And many Mac users stay on older than last two versions for stability and because they can't afford a new Mac. ;-)

Btw. is it possible, e.g. through some compiler flags, to target older versions of the Mac SDK on newer OS version? Because while one wants to compile low, we want to test on the latest versions as well.

I don't know.

[HannesWell]

Pawel from the EF-infra team provided us with the possible OS update options for the existing hardware in:

• https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/6621#note_5245497

At least for the x86 machine this is probably not sufficient.

He kindly also provided us with options to update the hardware in

• https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/6621#note_5246578

Btw. is it possible, e.g. through some compiler flags, to target older versions of the Mac SDK on newer OS version? Because while one wants to compile low, we want to test on the latest versions as well.

According to https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/cross_development/Configuring/configuring.html it's possible to set MACOSX_DEPLOYMENT_TARGET but I haven't tried it yet.

In fact it's already specified in: https://github.com/eclipse-platform/eclipse.platform.swt/blob/d87a932fb8e82bd892f06ed596a1f4c1145da88a/bundles/org.eclipse.swt/Eclipse%20SWT%20PI/cocoa/library/build.sh#L32

@Phillipus can you you check if the binaries produces for both mac archs adhere to that value?

Assuming it's possible to target older versions of the MacOS SDK my suggestion is to ask for the latest hardware with the latest software possible (for both arch) and then compile as low as we need/want to. @Phillipus or @sratz, @soethoff what do you think?

Please also feel free to comment on the help-desk issue.

[HeikoKlare]

Assuming it's possible to target older versions of the MacOS SDK my suggestion is to ask for the latest hardware with the latest software possible (for both arch) and then compile as low as we need/want to.

Fully agree with this proposal. Unfortunately, I cannot say anything about the possibility to target lower MacOS SDK when generating binaries.