Elmersolver execution time doubling when called with default input file!

[RexLinz]

When calling elmersolver from command line as "elmersolver case.sif" execution time almost doubles compared to calling "elmersolver" (using default case.sif). Total time is 67 respective 37 seconds.

I have logged the outputs for both cases (see attachments) and found the times to load the mesh (2.6 seconds) and executing the solver (~5x 1.7 seconds) is about the same, but assembly time increasing from 6.3 to 35.6 seconds. Model size is about 950.000 elements.

I expect that should not happen.

Unfortunately I can't share my mesh. But I should be able to create an example if you are not able to reproduce with any other model of similar size.

Windows 10 Home OS Build 19045.5608

Thanks!

Calling "elmersolver case.sif" case.log

Calling "elmersolver" default.log

Input file renamed to get accepted for upload case.sif.txt

[richb2k]

1. copy the test case at elmerfem/fem/tests/elstat
2. rename the sif file from elstatics.sif to case.sif
3. edit ELMERSOLVER_STARTINFO to contain 'case.sif'
4. add 'Mesh Levels = 2' to the simulation section

Calling elmersolver with case.sif took 23 seconds. Calling elmersolver without case.sif took 23 seconds.

Windows 11

[RexLinz]

Thanks richb2k. I did some tests on my desktop PC also running Windows 10 this morning (the one I reported has been from the notebook):

a) On the desktop PC time with and without case.sif added to the command line is about equal, both about 30 seconds for my mesh. What might be different in the installations? I'm not aware of installing something different.

b) adding 'Mesh Levels = 2' simulation time increased a lot but similar: 295 seconds calling without, 309 seconds with case.sif as argument. That's within regular tolerances I see when executing a solution several times.

Next I went back to my notebook, downloaded the test case you referenced. Execution time 18-20 seconds with or without argument case.sif. I compared the solver section and found some differences which might be interesting: My case.sif adds Stabilize = True, Optimize Bandwidth = True, Calculate Electrid Field = True and Calculate Capacitance Matrix = True, Capacitance Bodies = 5.

Adding Stabilize = True, Optimize Bandwidth = True, Calculate Electrid Field = True to the elmer test case decreased execution time a little bit, but equal calling with or without command line argument.

Disabling Calculate Capacitance Matrix, Stabilize and Optimize Bandwidth in my test case changed execution time, but **still about 25 seconds of difference calling with or without argument. ** So this looks like we could sort out these settings in the case.sif.

I'll have to create an example similar to mine I could share. Including a similar number of bodies (20) and also calculating capacitance matrix.

Thanks for the moment, I should be able to go on with this issue within the next few days.

[raback]

Using or not using "ELMERSOLVER_STARTINFO" should not affect the execution time. The piece of code that determines the name of the sif file is here:

 GotModelName = .FALSE.
 IF ( NoArgs > 0 ) THEN
   CALL GET_COMMAND_ARGUMENT(1, ModelName)
   IF( ModelName(1:1) /= '-') THEN 
     GotModelName = .TRUE.
     IF (NoArgs > 1) CALL GET_COMMAND_ARGUMENT(2, eq)
   END IF
 END IF

 IF( .NOT. GotModelName ) THEN
   OPEN( 1, File='ELMERSOLVER_STARTINFO', STATUS='OLD', IOSTAT=iostat )       
   IF( iostat /= 0 ) THEN
     CALL Fatal( 'MAIN', 'Unable to find ELMERSOLVER_STARTINFO, can not execute.' )
   END IF
   READ(1,'(a)') ModelName
   CLOSE(1)
 END IF

After the ModelName is defined the rest of the code is performed in the same way.

The "ELMERSOLVER_STARTINFO" was historically needed since passing command line arguments didn't work properly in parallel.

[RexLinz]

I can confirm ELMERSOLVER_STARTINFO is only required when I do not specify the name of the SIF file to process on the command line. But this does not change the behaviour on the notebook. Btw. I can't find information about the command line parameters or ELMERSOLVER_STARTINFO content in the ElmerSolver manual (PDF). ElmerGUI adds a second line with a number (not changing anything in my case if I delete).

[juharu]

Hi

Just a little historical note:

"ELMERSOLVER_STARTINFO" -file interface exists because in the far past (late 90's) there was no way (that I know) to give command line arguments to mpiexec/mpirun/etc executed programs. This changed later and the file is somewhat obsolete now, I guess.

Have fun!

Br, Juha

---

Lähettäjä: RexLinz @.> Lähetetty: maanantai 31. maaliskuuta 2025 10.57 Vastaanottaja: ElmerCSC/elmerfem @.> Kopio: Subscribed @.***> Aihe: Re: [ElmerCSC/elmerfem] Elmersolver execution time doubling when called with default input file! (Issue #654)

I can confirm ELMERSOLVER_STARTINFO is only required when I do not specify the name of the SIF file to process on the command line. But this does not change the behaviour on the notebook. Btw. I can't find information about the command line parameters or ELMERSOLVER_STARTINFO content in the ElmerSolver manual (PDF). ElmerGUI adds a second line with a number (not changing anything in my case if I delete).

— Reply to this email directly, view it on GitHubhttps://github.com/ElmerCSC/elmerfem/issues/654#issuecomment-2765424353, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACTOMSWXQ2MQAVK6JJIO5SL2XDYN3AVCNFSM6AAAAAB2C2UP2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONRVGQZDIMZVGM. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[RexLinz]RexLinz left a comment (ElmerCSC/elmerfem#654)https://github.com/ElmerCSC/elmerfem/issues/654#issuecomment-2765424353

I can confirm ELMERSOLVER_STARTINFO is only required when I do not specify the name of the SIF file to process on the command line. But this does not change the behaviour on the notebook. Btw. I can't find information about the command line parameters or ELMERSOLVER_STARTINFO content in the ElmerSolver manual (PDF). ElmerGUI adds a second line with a number (not changing anything in my case if I delete).

— Reply to this email directly, view it on GitHubhttps://github.com/ElmerCSC/elmerfem/issues/654#issuecomment-2765424353, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACTOMSWXQ2MQAVK6JJIO5SL2XDYN3AVCNFSM6AAAAAB2C2UP2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONRVGQZDIMZVGM. You are receiving this because you are subscribed to this thread.Message ID: @.***>

The information in this email may be confidential and is intended solely for the use of the individual or entity to whom it is intended. If you are not the intended recipient of this message, please delete the message and notify the sender immediately. For information on how we process personal data and our contact information, please see CSC's website: Privacyhttps://csc.fi/en/privacy

Tämän sähköpostin tiedot voivat olla luottamuksellisia ja ne on tarkoitettu yksinomaan sen henkilön tai yhteisön käyttöön, jolle ne on osoitettu. Jos et ole viestissä tarkoitettu vastaanottaja, tuhoa viesti ja ilmoita asiasta välittömästi viestin lähettäjälle. Tietoja henkilötietojen ja yhteystietojen käsittelystä löydät CSC:n verkkosivuilta: Tietosuojahttps://csc.fi/tietosuoja

[RexLinz]

Behavior getting even more interesting: I uninstalled and reinstalled ElmerFEM from the notebook. No change to before (the old and the new one has been 9.0, but showing a different compile date). I also tried all my other similar samples (electrostatic equation, 2D and 3D) and could not see that strange behavior on any of them.

Next I copied the full folder to another name in the same place side by side, deleted all files not absolutely necessary and ran "elmersolver case.sif". Big surprise: it took the same time as running without case.sif as argument. Then I copied the full folder to a different location on my drive (within the downloads folder) and there it shows the difference in execution time again!

I can run both alternating from identical folder content. The one running from the first copy is fine with "elmersolver case.sif", the other one takes about twice that time to execute.

Finally I renamed the folder running fast (FBE_test) to the initial name I faced that problem (FBE_volume) and it got slow. Renamed back to FBE_test and it is fast again. I can switch back and forth renaming the folder with 100% reproduction rate!

Anyone having an idea what's going on here?

[richb2k]

Sounds quite odd.

Please make sure that none of the folder names or paths have spaces. Elmersolver should tolerate spaces, but let's eliminate that as a possibility. BTW, if I remember right, Elmergrid does not tolerate spaces, so get used to not using spaces in paths or folder names.

Try adding 'Echo On' as the first line in the sif file. Run both cases and compare the solver log output using a diff program.

Posting two working examples would also help, of course.

[RexLinz]

While it might be irrelevant to the actual issue, I would like to start describing my workflow. My goal is to simulate a capacitive sensor setup including stray capacitance to the setup's frame and similar objects in the neighborhood. Thus I have to step a) the dielectric constant of the area of interest and b) evaluate changes of gemetry, e.g. placing the material probe, size of electrodes, ...

Geometric modeling To allow automated variations of the parameters I started with modeling the setup with GMSH using OpenCASCADE kernel. Using variables there it's quite easy to do changes to the geometry. That file is converted to an elmer compatible mesh by first running GMSH, next converting to elmer format using ELMERGRID. On the command line I'm using the batch file attached (renamed to .txt for upload) mesh3.txt.

FEM setup I'm using ElmerGUI to set up my solver, boundary conditions, output, ... Typically I start with a coarse mesh first to get the user interface responsive and check my simulation is working in general. Then I set a finer grid in the .geo input to GMSH, remesh and run the actual simulation on that, visualize results in ElmerVTK and check output files like capacitance matrix being realistic.

Parameter variation I take a copy of the SIF generated by ElmerGUI as template, replace the parameters I like to vary with special keywords like <<EPS>>. Using a program written in Gnu Octave (similar to Matlab) I do the main loop(s), depending on what to vary:

1. Outer loop: Using a template for the GEO file as well I'm replacing strings like <<OFFSETX>> with actual numbers and create the mesh calling GMSH and ELMERGRID using a system call.

2. Inner loop: Using the template for the SIF I'm replacing variables like <<EPS>> with actual numbers, write the SIF and call ELMERSOLVER via system call. Once ELMERSOLVER has completed simulation I read the required results, e.g. processing the file holding the capacitance matrix to extract the relevant values. NOTE I also tried to use MATC to vary material parameters but have not been successful doing so (looks like not changing the result, also overwriting the capacitance matrix calculated instead of appending results).

Finally these results are collected and nice graphs could be created showing influence of different imperfections of the setup.

While such simulations could run unassisted you can imagine running FEM for let's say 10 variations of geometry, 20 values for dielectic constant on a model with 1E6 unkowns each takes lot of time.

Current workaround I have found 2 useful ways to get the simulation fast independent of the location it is in the file system.

1. overwrite the SIF file generated by ELMERGUI (not prefered) with the modifications done by my loop and calling ELMERSOLVER without parameters
2. generate ELMERSOLVER_STARTINFO to point to a SIF file generated from my template, e.g. temp.sif. This ELMERSOLVER_STARTINFO could even be located in a different folder (e.g. parent folder) if you reference the correct path to the mesh folder within. Such the content of the ELMERGUI's project folder is not altered at all (just reading the mesh from there).

NOTE all above work fine even with spaces in the path to GMSH executable if you enclose the call to GMSH with quotes in the batch file.

[RexLinz]

Please make sure that none of the folder names or paths have spaces. Elmersolver should tolerate spaces, but let's eliminate that as a possibility. BTW, if I remember right, Elmergrid does not tolerate spaces, so get used to not using spaces in paths or folder names.

Spaces in the path look like being irrelevant for the current releases (also see my post above). I tried to move to different folders without change. I also expect to fail instead of changing execution time if the spaces are not handled correctly.

Try adding 'Echo On' as the first line in the sif file. Run both cases and compare the solver log output using a diff program. Adding Echo On did not give more relevant output (all the extra output is identical). But execution time changed in a strange matter.

Without 'Echo On' Calling "elmersolver > defaultEchoOff.log" defaultEchoOff.log total 33 seconds Calling "elmersolver case.sif > caseEchoOff.log" caseEchoOff.log total 60 seconds

With 'Echo On' Calling "elmersolver > defaultEchoOn.log" defaultEchoOn.log total 33 seconds Calling "elmersolver case.sif > caseEchoOn.log" caseEchoOn.log total 39 seconds

Difference in execution time is cut down ffrom 27 to 6 seconds with 'Echo On'!

And yes, I tried several times because I have surprised as well.

[raback]

Your timing results do not unfortunately make sense to me. For example "echo on/off" only affects the reading phase of the model. Unless you have included a huge x/y table or something similar the echoing should only affect on sub second level.

Are you running the different cases in a scripted fashion such that you launch the different runs almost at the same time? Even though you would have more cores than tasks the number of jobs running will have an effect as they are competing on memory and I/O operations. So to be able to compare timings exactly take care that you launch just one job at a time. Does this have any effect?

[raback]

Separate note: for capacitance matrix computation it is advisable to use p:2 or p:3 elements and special p-multigrid. You could also experiment with adaptive mesh refinement.

[RexLinz]

Your timing results do not unfortunately make sense to me. For example "echo on/off" only affects the reading phase of the model. Unless you have included a huge x/y table or something similar the echoing should only affect on sub second level.

What's surprising me is the calculation with Echo On is faster a lot (about 20 seconds) if called with argument compared to 'Echo Off'. Execution time is almost identical to calling with default argument (where Echo On did not change execution time).

Are you running the different cases in a scripted fashion such that you launch the different runs almost at the same time? Even though you would have more cores than tasks the number of jobs running will have an effect as they are competing on memory and I/O operations. So to be able to compare timings exactly take care that you launch just one job at a time. Does this have any effect?

For all the timings above I have called elmersolver from the command line and gave the Notebook time after boot up to complete all odd tasks windows is doing in the background after restarting, switched off synchronizing Onedrive, ... The only active programs are the command prompt and Notepad++ as editor. And I can repeat this results with a tolerance of maybe 1-2 seconds doing each of them multiple times.

Separate note: for capacitance matrix computation it is advisable to use p:2 or p:3 elements and special p-multigrid. You could also experiment with adaptive mesh refinement.

I don't know what you mean with p:2 or p:3 and p-multigrid. Could you provide a link where I could find infomation about that? Within GMSH I have set the mesh to different sizes in different areas respective boundaries, so the mesh is quite dense at the electrodes while coarse on the bounding box with 'infinite' boundary condition - interpolated grid size in between the bounderies. Anyway my results look realistic compared to measurements.

[raback]

Ok, this remains a puzzle to me.

See slides 9 and 14 for motivation and pointer to test case: https://wiki.f-si.org/images/7/71/Elmerfem-fsic2024.pdf

[RexLinz]

See slides 9 and 14 for motivation and pointer to test case: https://wiki.f-si.org/images/7/71/Elmerfem-fsic2024.pdf

Thanks for that interesting document. There is so much information around. I'll give a try once I find out where to set p:2 (looks like I have to add Element = "p:2" to the solver section in the sif) and if computation time is accaptable compared to improvment of accuracy with all the variations I have to do.

[raback]

Slide 14 compares p-elements from 1-7th degree considering the timing. Linear elements are not very competitive.

[richb2k]

Does running the command prompt as administrator have any effect?

You have probably already tried disabling any antivirus software, if so, what was the result?

[RexLinz]

Does running the command prompt as administrator have any effect?

No change in execution times.

You have probably already tried disabling any antivirus software, if so, what was the result?

The only antivirus protection active has been Windows standard protection. Switching all options available off there does not make a noticeable change in execution time (<1 sec, so within random changes).

... and the next curious thing: After all that tests done so far execution time decreased calling 'elmersolver case.sif' decreased to about 41 seconds (with or without Echo On), while calling with default input stays at about 33 second. Looking into the details I see the speedup is within the initial assembly decreasing from about 35 seconds during the last days to 14 seconds with 'elmersolver case.sif'. I checked windows update history, no updates have installed within the last week, just the daily update of windows defender database.

[RexLinz]

See slides 9 and 14 for motivation and pointer to test case: https://wiki.f-si.org/images/7/71/Elmerfem-fsic2024.pdf

I experimented a little bit according to what I have found there.

First of all I had to change from Procedure = "StatElecSolve" "StatElecSolver" ElmerGUI has inserted to Procedure = "StatElecSolveVec" "StatElecSolver"_ to be able to add Element = "p:2" without errors.

That did not solve within >30 Iterations (I did not measure the time it has been running). For testing I set Element = "p:1", that took almost 500 seconds to solve, a time I could not spend per run for my parameter variations. Results in the capacitance matrix different by less than 1%. So not worth to go that way. I also tried to copy the solver section from the sif on https://github.com/ElmerCSC/elmer-elmag/tree/main/CapacitanceOfTwoBalls, but that raised an error processing the sif. Obviously there is more to change to get it working.

At the moment I don't have the time to focus on that, I might be back on that later so thanks for pointing me to that PDF.