Native support for Eclipse SDKs on Windows Arm64

[chirontt]

Update: now that adding support for Windows on Arm64 is under way for various Eclipse repositories, here's the central TODO list to keep track of various tasks done and/or under way:

• [x] java-native-access/jna#1466
• [x] add support for Windows on Arm64 to eclipse.platform.swt submodule
  • [x] eclipse-platform/eclipse.platform.swt#1048
  • [x] eclipse-platform/eclipse.platform.swt#1045
  • [x] eclipse-platform/eclipse.platform.swt#1137
  • [x] eclipse-platform/eclipse.platform.swt#1158. This is temporary measure to have successful build; to be reverted later.
    • [x] eclipse-platform/eclipse.platform.swt#1222
  • [x] eclipse-platform/eclipse.platform.swt#1172
• [x] add support for Windows on Arm64 to equinox submodule
  • [x] eclipse-equinox/equinox#559
  • [x] eclipse-equinox/equinox#564
  • [x] eclipse-equinox/equinox#575
  • [x] eclipse-equinox/equinox#603
  • [x] eclipse-equinox/equinox#621
• [x] add support for Windows on Arm64 to rt.equinox.p2 submodule
  • [x] eclipse-equinox/p2#492
• [x] add support for Windows on Arm64 to eclipse.platform submodule
  • [x] eclipse-platform/eclipse.platform#1311
  • [x] eclipse-platform/eclipse.platform#1350
  • [x] eclipse-platform/eclipse.platform#1351
  • [x] eclipse-platform/eclipse.platform#1394
• [x] add support for Windows on Arm64 to eclipse.platform.ui submodule
  • [x] eclipse-platform/eclipse.platform.ui#1789
  • [x] eclipse-platform/eclipse.platform.ui#1787
• [x] add support for Windows on Arm64 to eclipse.platform.releng.aggregator
  • [x] eclipse-platform/eclipse.platform.releng.aggregator#1942
  • [x] eclipse-platform/eclipse.platform.releng.aggregator#1953
  • [x] eclipse-platform/eclipse.platform.releng.aggregator#1917
  • [x] eclipse-platform/eclipse.platform.releng.aggregator#2017
• [x] eclipse-platform/www.eclipse.org-eclipse#169
• [ ] eclipse-packaging/packages#162

---

I'm trying to build the Eclipse SDKs for Windows 11 on Arm64 (WoA), using this repo. I hope this is the right repo to start with, which is the umbrella repo for all others.

The environment triplet I use for it is win32/win32/aarch64, following the existing win32/win32/x86_64 triplet as a guide.

After re-compiling many *.exe and *.dll native files for WoA in various repos (from existing C code mostly unchanged), and adding many new win32.aarch64 folders (most of them are duplicates of the existing win32.x86_64 ones), I've managed to build successfully this aggregator repo, and got the following new zip files in eclipse.platform.releng.tychoeclipsebuilder/eclipse.platform.repository/target/products (in addition to other supporting os/arch zip files there):

org.eclipse.platform.ide-win32.win32.aarch64.zip
org.eclipse.sdk.ide-win32.win32.aarch64.zip

After unzipping the org.eclipse.sdk.ide-win32.win32.aarch64.zip file, I discover that, in its eclipse/plugins folder, the following 6 bundles are missing:

org.eclipse.core.filesystem.win32.aarch64_<version>.jar
org.eclipse.core.net.win32.aarch64.source_<version>.jar
org.eclipse.core.net.win32.aarch64_<version>.jar
org.eclipse.core.resources.win32.aarch64_<version>.jar
org.eclipse.equinox.security.win32.aarch64.source_<version>.jar
org.eclipse.equinox.security.win32.aarch64_<version>.jar

When compared to the org.eclipse.sdk.ide-win32.win32.x86_64.zip distribution, those bundles (of relevant x86_64 arch) do exist there. So somehow they got missing from the final zip file for aarch64 during the build.

The missing bundles are crucial for the Eclipse SDK to properly start up, but I have no idea how they got missed out from the build artifacts. I think I must have skipped over some crucial file(s) somewhere which produced this missing result.

Is there a file somewhere in the repo to indicate which bundles to be included (in the plugins folder of the zip file) for each SDK?

PS. The missing bundles are actually produced in their own Maven modules during the build, in the eclipse.platform & equinox submodules. Once I manually add 4 of them (the other 2 are source bundles) to the plugins folder, and also add their missing entries to the configuration\org.eclipse.equinox.simpleconfigurator\bundles.info file, starting it up with eclipse.exe works nicely!

PPS. Especially and crucially for WoA, eclipse.exe can only start up after I add the -Dorg.osgi.framework.os.name=win32,win entry to the eclipse.ini file, to help it resolve the critical com.sun.jna bundle at runtime.

[merks]

Did you update the features to include the new fragments?

[chirontt]

@merks thanks for the tip. Your picture is worth more than a thousand words to me. Now I can build the Eclipse SDKs for WoA and they work!

Next is where/what is the repo to build the Eclipse IDE (which should include other tools like JGit/EGit, m2e, etc.) for WoA?

[merks]

The repository used to build the IDE are composed from many projects. There's information about that here:

https://ci.eclipse.org/simrel/

This file has information about all the projects involved:

https://git.eclipse.org/c/simrel/org.eclipse.simrel.build.git/tree/simrel.aggran

Quite of a few of the project has Oomph setups that you can use to setup an an IDE; many of them you'd have to do manually.

[mickaelistria]

I don't think you'll need to change many other projects. The very vast majority of them wouldn't have Platform specific artifacts, and I don't have the impression fixing those is an immediate requirement for you. As @merks pointed, you might want to make the SimRel build capable of including the win32/aarch64 artifacts; adding a relevant line at https://git.eclipse.org/c/simrel/org.eclipse.simrel.build.git/tree/simrel.aggr#n74 . But SimRel references p2 repositories, so you'll need to tweak it to reference a p2 repository that contains the win32/aarch64 SDK, which might not be easy. Then, from SimRel, the "Eclipse IDE" is build with https://git.eclipse.org/c/epp/org.eclipse.epp.packages.git/ ; there are some pom.xml and .product files in here you'll also need to modify for this new platform to be considered.

But this is a multi-step process. My advice would be that -if you're willing to make your work sustainable and to contribute it to Eclipse Project-, now that you have a build working, you find a way to get a win32/aarch64 machine contributed to the build farm and contribute your changes to the Eclipse Platform/SDK project to get this Platform supported. Going to "official" SDK and p2 repo will already be a major milestones and will make it easier to cascade later on to SimRel/EPP...

[chirontt]

As to the problem with JNA bundle in WoA:

PPS. Especially and crucially for WoA, eclipse.exe can only start up after I add the -Dorg.osgi.framework.os.name=win32,win entry to the eclipse.ini file, to help it resolve the critical com.sun.jna bundle at runtime.

I've raised an issue with the JNA project: java-native-access/jna#1465

[chirontt]

@mickaelistria My Arm64 box is an entry-level, slow box, suitable for some light, manual testing only. I had started building this repo with this Arm64 box, and it was a big exercise in patience. I eventually gave it up and switched to building on a Linux (x64) box instead. It takes around 1 hour to build this repo on the Linux box without running the tests (i.e. with -DskipTests=true). The Arm64 box becomes a test machine for some manual testing of the resulting SDKs. So I don't think it's a good addition to your build farm.

The good news is that you don't need an Arm64 box to build these SDKs for WoA. I can build and produce all SDKs for all os/arch (WoA included) with my Linux (x64) box, so your existing build farm can build the WoA distribution too, once my work on WoA are accepted in to the official repos. The recompiled binaries for Arm64 would be checked in to the repos like other existing binaries currently for other os/arch, so that the build wouldn't depend on any platform-specific tools to run.

From the look of it, building the Eclipse IDE for WoA is more than I can chew at the moment, so for now I will concentrate on submitting PRs for this WoA work to various repos (9 total, including this repo), to get the official Platform/SDK build started for WoA!

[mickaelistria]

@chirontt OK, thanks for the details. So if you have things that you could already contribute, please do so and we'll see what we can merge easily. But at some point, to guarantee good enough quality, we'll need to have the SDK tests running automatically on a win32/aarch64 machine. Without it, then the project cannot prevent from potential regressions easily, and we basically cannot provide good enough support ot make it reliable; which IMO kind of question the value of hosting this specific code in the Eclipse project instead of eg in some fork of yours.

[chirontt]

Can this aggregator build be run for Windows on Arm64 only, i.e. for win32/win32/aarch64 environment triplet only? How should it be configured?

If so, I would try to build just for this WoA platform on my Arm64 box, including running the tests, to see how it would fare in producing the SDKs for WoA. I can't give any promise at the moment, though.

BTW, I've started sending out the PRs for this WoA work.

[mickaelistria]

Some of the CI issues such as the ones about not found bundles in https://ci.eclipse.org/equinox/job/equinox/job/PR-121/1/console are caused by the missing win32/win32/aarch64 declaration in the parent pom: https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/blob/master/eclipse-platform-parent/pom.xml#L254 . Please try adding the environment there and if build is fine, submitting a PR for that. If you want to try a local build skipping other environments, you can comment them out.

[akurtakov]

We can have the bundles but can't include them in features/products until we have the builder to ensure that they are rebuild on source changes.

[chirontt]

Sorry for the confusion in my partial submission of PRs. I ran out of time yesterday, and only managed to submit the PRs for eclipse-equinox repos.

Now I've done all other PRs for eclipse-platform repos, to complete this WoA submission to mainline.

My guess is that all these PRs need to go in together for a successful build, as they are inter-related, but you may have better idea about them.

[akurtakov]

I'll repeat it: We can not include native win32 aarch fragments in features unless we have a builder in our JIPPs allowing to rebuild them on e.g. source change in SWT. Until this happens we can only have sources that can't end up in deliverables.

[akurtakov]

@daliasheasha You was interested in Eclipse on Windows AArch64. Maybe you can help with some builder?

[chirontt]

@akurtakov I'll resubmit the PRs if you can show me what can be submitted now, and what should be submitted later when the build farm would have a builder available to (re)build Windows on Arm64 artifacts. This requires in-depth knowledge of the overall build requirements of all the repos, knowledge that I don't have.

Although I've withdrawn all the PRs, for the records, here was the list of them again that contains all of the WoA work in my forks:

https://github.com/eclipse-equinox/equinox/pull/121 https://github.com/eclipse-equinox/equinox.binaries/pull/2 https://github.com/eclipse-equinox/p2/pull/147 https://github.com/eclipse-platform/eclipse.platform/pull/189 https://github.com/eclipse-platform/eclipse.platform.releng/pull/117 https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/pull/595 https://github.com/eclipse-platform/eclipse.platform.swt/pull/413 https://github.com/eclipse-platform/eclipse.platform.swt.binaries/pull/16 https://github.com/eclipse-platform/eclipse.platform.ui/pull/365

The branches in my forks for them are still available, to be resubmitted to mainline if needed. From my point of view, submitting them all together would lessen the chance of build breakage, but you may have better idea.

I know that all changes in those PRs would produce a successful Eclipse SDK for Windows on Arm64 platform, but separating them to do partial submits to mainline is a tough job for me, not to mention (God forbid!) build breakage. I'm willing to create new branches in my forks to submit the PRs again, if you can give me a precise list of what can be accepted, or wouldn't be needed (natives?), out of the above (withdrawn) PRs.

[chirontt]

FYI, my fork of this umbrella repo, for Windows-on-Arm64 support, is now available:

https://github.com/chirontt/eclipse.platform.releng.aggregator

The fork includes links to other forked submodules, to make up my whole Windows-on-Arm64 work.

To build it is like normal (it can be built on any OS, be it Linux, Windows, or MacOS (haven't actually tried this myself)):

> git clone --recurse-submodules https://github.com/chirontt/eclipse.platform.releng.aggregator
> cd eclipse.platform.releng.aggregator
> mvn verify -DskipTests=true

The eventual build artifacts for all Eclipse SDKs of all os/arch will be produced in the eclipse.platform.releng.tychoeclipsebuilder/eclipse.platform.repository/target/products/ directory.

And the specific artifact of my interest would be org.eclipse.sdk.ide-win32.win32.aarch64.zip, which should be created in that directory. Note that, in order to run this artifact successfully on a WoA machine, you must apply the workaround I mentioned above, for Eclipse to successfully resolve the com.sun.jna bundle at startup.

I will try to keep this fork tracking the upstream repo closely, to keep it from becoming outdated, perhaps by merging it with upstream repos every weekend.

[akurtakov]

I did one of them in https://github.com/eclipse-platform/eclipse.platform.ui/pull/365#pullrequestreview-1126289248

[akurtakov]

@waynebeaton Maybe you can try foundation MS connections for having win/aarch64 machine on the build farm?

[SyntevoAlex]

There's no need to have win/aarch64 machine to build bundles, see https://github.com/eclipse-platform/eclipse.platform.swt/pull/413#issuecomment-1270658605

Without such machine, we won't be able to run tests against built binaries, but this is only a minor issue: it's unlikely to have any new bugs compared to x64 bundle.

[merks]

I expect we'll want to be able to test that this stuff actually works. Also, if we want SimRel/EPP to deliver these things, will need a JustJ JRE for which we need to run jlink natively. If no one is willing to commit to providing hardware, what does that say about depth of the commitment?

[akurtakov]

I said it already - it's fine to have sources for anyone willing to work on it. But it's not fine to ship any user binaries without being able to at least smoketest it. We have went through such hoops with linux/[aarch64|ppc64le] - binaries produced were not even starting in real world until smoketesting infrastructure was present.

[karianna]

Hi folks - Martijn from Microsoft (and Eclipse Adoptium) here. I'm wondering if some of the Windows on ARM hosts we have at Eclipse Adoptium could be utilized for this project? I'll check with George Adams when he's back from leave and see what we can do.

[akurtakov]

@karianna so is there progress on your arm hosts being available for releng jipp?

[karianna]

An option is Project Volterra - https://blogs.windows.com/windowsdeveloper/2022/10/24/available-today-windows-dev-kit-2023-aka-project-volterra/

Not sure if we can do the Adoptium loan - CC @gdams

[akurtakov]

I'm closing this one as there is clearly no interest from anyone to step up and do some work. If/when one does please reopen or create new issue.

[chirontt]

FYI: In the meantime, I'm putting out unofficial Eclipse SDK builds for Windows on Arm64 in my fork.

I guess my fork will exist until Eclipse starts officially supporting Windows on Arm64 in its IDE releases.