Custom test documentation is very sparse - can we have more?

[jkwmoore]

It would be great if the documentation for creating custom tests was a little less sparse with some examples on how to build a test with multiple outputs and / or continous data for a scatter.

e.g. Benchmark time and load time from the following: https://github.com/jkwmoore/sheffield_hpc_custom_phoronix_tests/blob/main/Benchmarks/fluent-test-benchmark-1.0.0/sourcefiles/fluent-test-benchmark-files/fluent-test-benchmark.sh

[michaellarabel]

Unfortunately the lack of documentation at this time comes down to resource constraints and thus for new contributors largely relying upon looking at existing test profiles as the main source of documentation.

I am happy to answer any specific questions as well. Or even if you have a test script / test case you are looking to upstream to PTS/OpenBenchmarking.org, I am happy to get it better integrated/working with the test profile if you at least have a test script building/running the test and provide guidance on the result(s) to extract. Is that script you are pointing out in such a state already? If so would be happy to look at it and get any integration points finished up for getting the test working.

For multiple outputs, I'd recommend the likes of looking at fio test profile that outputs throughput and latency or the likes of hpcc and others.

[jkwmoore]

I'm still fiddling with the test at the moment but the folder to look at will be https://github.com/jkwmoore/sheffield_hpc_custom_phoronix_tests/tree/main/Benchmarks/fluent-test-benchmark-1.0.0

I've been looking at the examples given as the inspiration for this and other future benchmarks I have in mind and the above should be in a working state for anyone with fluent available in their PATH (via module loading or otherwise.)

When I am happy with the above my plan was to format it nicely and PR it to you. At present it is simply extracting the total time via sensor - but in future I would be looking at adding the load time and benchmark time (thus a total) and potentially looking at parsing a time per iteration.

Example Fluent output below (... waiting for a benchmark to finish):

> ; Run for 1000 iterations
/so it 1000

  iter  continuity  x-velocity  y-velocity  z-velocity           k     epsilon     time/iter
     1  1.0000e+00  8.2594e-03  0.0000e+00  0.0000e+00  2.1325e-01  6.8178e-01  0:05:55  999
     2  1.0000e+00  1.1924e-02  2.0250e-06  6.4945e-06  1.1148e-01  6.0595e-01  0:05:41  998
     3  1.0000e+00  1.4560e-02  4.5356e-06  1.0219e-05  5.1625e-02  4.8896e-01  0:05:32  997
     4  1.0000e+00  1.8771e-02  8.2189e-06  1.3379e-05  3.8865e-02  3.9469e-01  0:05:14  996
     5  1.0000e+00  2.1336e-02  1.1128e-05  1.6086e-05  3.7431e-02  3.0294e-01  0:05:00  995
     6  1.0837e+00  2.2317e-02  1.2208e-05  1.7913e-05  3.9314e-02  2.3724e-01  0:04:49  994
     7  1.1229e+00  2.2494e-02  1.1823e-05  1.8382e-05  4.0441e-02  2.1013e-01  0:04:46  993
     8  1.1465e+00  2.2213e-02  1.0766e-05  1.7803e-05  4.0786e-02  2.1789e-01  0:04:36  992

Uploaded to Openbenchmarking here:

https://openbenchmarking.org/result/2012246-NE-FTBR1280918

Some bits will need more documentation/signposting - e.g. $FLUENT_OPT_ARGS and what the different mesh versions actually mean. I'm likely to get a bit more involved with getting this test working in the new year mind you and I am more than happy to contribute / help out with custom phoronix test docs as and when I can fit this in.

There's also quite a lot of potential in terms of several different Fluent models - specifically the ones that ANSYS themselves supply rather than this Sheffield Uni specific one I am testing with.

[michaellarabel]

Hi @jkwmoore just seeing if that test profile is in good enough shape now for upstreaming or if any other lingering questions/issues remain? Thanks.

[jkwmoore]

I've not had time to look into analysis of the generated log files but as it stands I think it is relatively good enough to give you a total time to finish.

I need to look back into this and see if it needs a bit of extra work as it is probably useful to extract the load time as well as the execution time.