temurin-build icon indicating copy to clipboard operation
temurin-build copied to clipboard
verified publisher icon adoptium

AIX builds failing on ar commands

Open sxa opened this issue 4 years ago • 5 comments

Thank you for reporting a broken build! Please fill in the following details:

  • Jenkins link to the failing log: For example, https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-aix-ppc64-hotspot/1002/consoleFull - started failing with bulid 1002 on the 28th September
  • Which platforms (OS+arch) are affected: AIX/ppc64
  • If you look at the historic logs, does the problem appear to be specific to one machine? No - happens on both
  • Link to any upstream project bug report (e.g. OpenJ9/JBS) n/a but it may be related to the addition of the static-libs target in https://github.com/adoptium/temurin-build/pull/2725 (FYI @jerboaa)

sxa avatar Sep 30 '21 10:09 sxa

At @aixtools' suggestion I've set OBJECT_MODE=64 on the jenkins agent configuration for the two AIX build machines to see if that resolves the problem.

sxa avatar Sep 30 '21 12:09 sxa

Posting the failure here for posterity:

20:25:53  	Use the -X option to specify the desired object mode.
20:25:53  ar: 0707-126 /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-aix-ppc64-hotspot/workspace/build/src/build/aix-ppc64-normal-server-release/support/native/java.management/libmanagement/static/VMManagementImpl.o is not valid with the current object file mode.

jerboaa avatar Sep 30 '21 12:09 jerboaa

If this keep failing, we can disable static-libs for AIX. It wouldn't be a supported OS for Graal VM/Mandrel builds anyway.

jerboaa avatar Sep 30 '21 12:09 jerboaa

Seems to be working ok, but that's a fair point about the builds not being supported for that use case. For some reason after enabling it the builds have got very slow (2h15 vs 40 minutes), but unclear if that's just a one off.

I'm assuming the use of OBJECT_MODE=64 isn't causing any other side effects. If it turns out to be the correct solution (relative to disabling the static libs on AIX, or any other option we try) then we should put that into the aix.sh in this repository rather than in the jenkins agent configuration.

sxa avatar Sep 30 '21 16:09 sxa

This issue appears to have been seen again on test-osuosl-aix72-ppc64-6. The Jenkins environment variables have been checked, and it seems OBJECT_MODE was never set for this machine (nor test-osuosl-aix72-ppc64-5).

The OBJECT_MODE variables have now been set on both of these machines, and I ask that the jenkins agent be reset at the earliest opportunity.

Also, this error is in the same job output: "1586-495 (U) SYSTEM LIMIT EXCEEDED: Too many processes are active."

So please also review the ulimits on these machines, as well as the mechanism for setting both ulimits and jenkins environment variables.

adamfarley avatar Jun 27 '23 10:06 adamfarley