rcps-buildscripts
rcps-buildscripts copied to clipboard
UCL-RITS
Install & Repair Request: ANSYS 2023.R1/2, 2024.R1
Application: ANSYS 2023.R1 and R2
Cluster: Myriad & Kathleen at least
Description: does fluid dynamics and other macroscopic physics simulations
License: proprietary
Ticket number: IN:00000000
ANSYS 2023.R1 is installed, but currently has some remaining problems: it seems the MPI wrappers do not correctly export the environment? This is causing errors where the Fluent binary cannot find either libmpi_wrapper.so or libansysfluidshdfsdk.so (or presumably other libraries it would load after that).
ANSYS 2023.R2 is not currently installed. I assume that this problem is likely, but not certain, to show up in 2023.R2 as well.
To reproduce the problem:
fluent 3d -g -mpitest
You may see Tcl errors, but also this error:
/shared/ucl/apps/ANSYS/2023.R1/v231/fluent/fluent23.1.0/multiport/mpi_wrapper/test/lnamd64/test: error while loading shared libraries: libmpi_wrapper.so: cannot open shared object file: No such file or directory
If you run Fluent without the option to test, the problem manifests as this error instead:
/shared/ucl/apps/ANSYS/2023.R1/v231/fluent/fluent23.1.0/lnamd64/3d_node/fluent_mpi.23.1.0: error while loading shared libraries: libansysfluidshdfsdk.so: cannot open shared object file: No such file or directory
Here are the Tcl errors I see, but they only appear on Kathleen; maybe a difference in Tcl libs installed in the OS.
$ fluent 3d -g -mpitest
/shared/ucl/apps/ANSYS/2023.R1/v231/fluent/fluent23.1.0/bin/fluent -r23.1.0 3d -g -mpitest
application-specific initialization failed: Can't find a usable init.tcl in the following directories:
/home2/polyflowbuilds/buildagents/milpolybld01/work/1210bc2a0080eb69/Tcl-tk/8_5_11/linx64/Release/tcltk/lib/tcl8.5 /usr/lib/tcl8.5 /lib/tcl8.5 /usr/library /library /tcl8.5.11/library /tcl8.5.11/library
This probably means that Tcl wasn't installed properly.
I think the setup for the MPI environment is done in:
/shared/ucl/apps/ANSYS/2023.R1/v231/CEI/apex231/machines/linux_2.6_64/multiport/mpi_wrapper/bin/mpirun.fl
but I haven't had time to look much further into it yet.
@balston noted that it is necessary to use a non-default MPI, instead, e.g.:
fluent 3ddp -t$NSLOTS -mpi=intel2018 -cnf=$TMPDIR/machines -g < your-input-file
Is it feasible to change this to be the new default?
The MPI test using this option passes successfully within a single node:
fluent 3d -g -mpi=intel2018 -t 2 -mpitest
ANSYS Mechanical from 2023.R1 doesn't work on Kathleen (and possibly Myriad) as it picks up the wrong version of Tcl (the one from /usr/bin). This causes jobs to fail with errors like:
application-specific initialization failed: Can't find a usable init.tcl in the following directories:
/usr/share/tcl8.5 /home2/polyflowbuilds/buildagents/milpolybld01/work/1210bc2a0080eb69/Tcl-tk/8_5_11/linx64/Release/tcltk/lib/tcl8.5 /usr/lib/tcl8.5 /lib/tcl8.5 /usr/library /library /tcl8.5.11/library /tcl8.5.11/library
/usr/share/tcl8.5/init.tcl: version conflict for package "Tcl": have 8.5.11, need exactly 8.5.13
version conflict for package "Tcl": have 8.5.11, need exactly 8.5.13
while executing
"package require -exact Tcl 8.5.13"
(file "/usr/share/tcl8.5/init.tcl" line 18)
invoked from within
"source /usr/share/tcl8.5/init.tcl"
("uplevel" body line 1)
invoked from within
"uplevel #0 [list source $tclfile]"
This probably means that Tcl wasn't installed properly.
ANSYS 2024.R1 is now available so probably worth attempting to install this version.
This should be possible. I'll download the ANSYS 2024 installers from behind the license portal and upload them to Kathleen.
I've downloaded the ANSYS 2024.R1 installer archives to my laptop (all 45 GB worth!), done checksum checks and will upload them to Kathleen tomorrow.
Thew ANSYS 2024.R1 installer archives are now in:
/shared/ucl/apps/ANSYS/installers/2024.R1
on Kathleen and their checksums have been checked against the ANSYS Inc published ones.
The install script is running as ccspapp:
./ANSYS-2024R1_install 2>&1 | tee ~/Software/ANSYS/ANSYS-2024R1_install.log-1
and its running really slowly.
The install hung during the ANSYS Electromagnetics installer yesterday so I'm re-running from this one now.
The ANSYS Electromagnetics installer ran to completion yesterday afternoon and I've now run the additional update install as well. I will now add a module file and run some test jobs.
my ANSYS CFX test job has run successfully over 80 cores at the second attempt. Submitting a Fluent test job next.
Fluent test job ran successfully overnight. Now submitting a AnsysEM test job.
ANSYS Mechanical test job failing with:
OMP: Error #100: Fatal system error detected.
OMP: System error #22: Invalid argument
forrtl: error (76): Abort trap signal
suggestions online suggest switch to an alternative MPI. ANSYS now provide an OpenMPI so will try that.