ProjectPhysX
Report your benchmark results here!
You are welcome to report your benchmark results for the FP32/FP16S/FP16C accuracy levels here. Especially numbers for AMD GPUs are desired for GCN/RDNA/RDNA2 architectures. Thank you!
I'd love to add to the benchmarks list. I've got two questions:
- I want to benchmark a dual Epyc system (so specifically the CPUs actually). How would I do that (under Windows, but Linux would also be fine), if I have a GPU installed? It always automatically detects the GPU when running the benchmark “releases”.
- How to post the benchmarks? Just copy the console output in here?
Cheers!
Hi ibonito1,
OpenCL support on EPYC CPUs is a bit difficult as these are not officially supported by AMD. Being x86-64, they should work with the Intel OpenCL CPU Runtime though, or alternatively with POCL. Fingers crossed! To run on a specific device, in the console run ./FluidX3D.exe 2 (on Linux) or FluidX3D.exe 2 (on Windows), to select device with ID 2 for example. You can just copy the console output here.
Regards, Moritz
C-Dub2022 thank you very much for the RX 580 benchmark! If you can post the FP16S and FP16C benchmarks as well, I'll add them to the readme!
Hopefully this is helpful. Let me know if there is anything else I can do.
Could you please tell me how to open the visualized interface of the flow domain as you said in the readme file? You said input the 2 can turn on the velocity field, but it does not work in the benchmark case. How can I generate pictures like you prensent on twitter ?
Hi dongwang22,
thanks for the benchmark! For the visual interface, uncomment #define WINDOWS_GRAPHICS and comment #define BENCHMARK in src/defines.hpp, and uncomment for example the Taylor-Green setup in src/setup.cpp. Then compile and you should see the graphical interface where you can toggle rendering modes with keys 1/2/3/4. To generate videos, see the other setups: basically make a C++ loop and repeatedly do some LBM time steps and render images with the corresponding methods of the LBM class.
Regards, Moritz
AMD 5700 XT
|----------------.------------------------------------------------------------| | Device ID 0 | gfx1010:xnack- | |----------------'------------------------------------------------------------| |----------------.------------------------------------------------------------| | Device ID | 0 | | Device Name | gfx1010:xnack- | | Device Vendor | Advanced Micro Devices, Inc. | | Device Driver | 3444.0 (PAL,LC) | | OpenCL Version | OpenCL C 2.0 | | Compute Units | 20 at 1905 MHz (2560 cores, 9.754 TFLOPs/s) | | Memory, Cache | 8176 MB, 16 KB global / 64 KB local | | Buffer Limits | 6949 MB global, 7116390 KB constant | |----------------'------------------------------------------------------------| | Info: OpenCL C code successfully compiled. | |-----------------.-----------------------------------------------------------| | Grid Resolution | 256 x 256 x 256 = 16777216 | | LBM Type | D3Q19 SRT (FP32/FP32) | | Memory Usage | CPU 272 MB, GPU 1488 MB | | Max Alloc Size | 1216 MB | | Time Steps | 10 | | Kin. Viscosity | 1.00000000 | | Relaxation Time | 3.50000000 | | Reynolds Number | Re < 148 | |---------.-------'-----.-----------.-------------------.---------------------| | MLUPs | Bandwidth | Steps/s | Current Step | Time Remaining | | 1366 | 209 GB/s | 81 | 9996 60% | 0s | |---------'-------------'-----------'-------------------'---------------------| | Info: Peak MLUPs/s = 1368 |
|----------------.------------------------------------------------------------| | Device ID 0 | gfx1010:xnack- | |----------------'------------------------------------------------------------| |----------------.------------------------------------------------------------| | Device ID | 0 | | Device Name | gfx1010:xnack- | | Device Vendor | Advanced Micro Devices, Inc. | | Device Driver | 3444.0 (PAL,LC) | | OpenCL Version | OpenCL C 2.0 | | Compute Units | 20 at 1905 MHz (2560 cores, 9.754 TFLOPs/s) | | Memory, Cache | 8176 MB, 16 KB global / 64 KB local | | Buffer Limits | 6949 MB global, 7116390 KB constant | |----------------'------------------------------------------------------------| | Info: OpenCL C code successfully compiled. | |-----------------.-----------------------------------------------------------| | Grid Resolution | 256 x 256 x 256 = 16777216 | | LBM Type | D3Q19 SRT (FP32/FP16S) | | Memory Usage | CPU 272 MB, GPU 880 MB | | Max Alloc Size | 608 MB | | Time Steps | 10 | | Kin. Viscosity | 1.00000000 | | Relaxation Time | 3.50000000 | | Reynolds Number | Re < 148 | |---------.-------'-----.-----------.-------------------.---------------------| | MLUPs | Bandwidth | Steps/s | Current Step | Time Remaining | | 3253 | 250 GB/s | 194 | 9988 80% | 0s | |---------'-------------'-----------'-------------------'---------------------| | Info: Peak MLUPs/s = 3253 |
|----------------.------------------------------------------------------------| | Device ID 0 | gfx1010:xnack- | |----------------'------------------------------------------------------------| |----------------.------------------------------------------------------------| | Device ID | 0 | | Device Name | gfx1010:xnack- | | Device Vendor | Advanced Micro Devices, Inc. | | Device Driver | 3444.0 (PAL,LC) | | OpenCL Version | OpenCL C 2.0 | | Compute Units | 20 at 1905 MHz (2560 cores, 9.754 TFLOPs/s) | | Memory, Cache | 8176 MB, 16 KB global / 64 KB local | | Buffer Limits | 6949 MB global, 7116390 KB constant | |----------------'------------------------------------------------------------| | Info: OpenCL C code successfully compiled. | |-----------------.-----------------------------------------------------------| | Grid Resolution | 256 x 256 x 256 = 16777216 | | LBM Type | D3Q19 SRT (FP32/FP16C) | | Memory Usage | CPU 272 MB, GPU 880 MB | | Max Alloc Size | 608 MB | | Time Steps | 10 | | Kin. Viscosity | 1.00000000 | | Relaxation Time | 3.50000000 | | Reynolds Number | Re < 148 | |---------.-------'-----.-----------.-------------------.---------------------| | MLUPs | Bandwidth | Steps/s | Current Step | Time Remaining | | 3044 | 234 GB/s | 181 | 9992 20% | 0s | |---------'-------------'-----------'-------------------'---------------------| | Info: Peak MLUPs/s = 3049 |
I tried a 6900XT, but the score is lower than anticipated. The max bandwidth seems to be limited to 300GB/s, although GPUZ says it's connected via PCIe 4.0 16x and should top out at 512GB/s. The GPU clock is at 2540MHz and the memory clock at 2000MHz. GPU and memory controller loads are at 100%.
With the 3D Taylor-Green model and FP32/FP16S, the MLUPs/s and the bandwidth go through the roof. I'll try some other models, too. FP32/FP32 goes up to 2400 MLUPs/s and 370GB/s, with FP32/FP16C it's 9000 MLUPs/s and 700GB/s.
Hi gittigittibangbang, thanks for the benchmarks! Efficiency is ~60% which is typical for the AMD GPUs. Performance is limited by VRAM bandwidth only, and the RX 6800 would presumably perform exactly the same. The benchmark setup is a 256³ box, that fills 1.5GB (FP32) or 0.9GB (FP16) of VRAM. The large infinity cache (128MB) is only an insignificant fraction of that so does not significantly boost performance. With a smaller 128³ box however, which only fills 186MB (FP32) or 76MB (FP16), almost the entire grid fits in the cache and effective bandwidth is much larger.
Vega 8 in R7 4750G |----------------.------------------------------------------------------------| | Device ID 0 | gfx90c | | Device ID 1 | gfx90c | | Device ID 2 | gfx90c | |----------------'------------------------------------------------------------| |----------------.------------------------------------------------------------| | Device ID | 0 | | Device Name | gfx90c | | Device Vendor | Advanced Micro Devices, Inc. | | Device Driver | 3380.6 (PAL,HSAIL) | | OpenCL Version | OpenCL C 2.0 | | Compute Units | 8 at 2100 MHz (512 cores, 2.150 TFLOPs/s) | | Memory, Cache | 26899 MB, 16 KB global / 32 KB local | | Buffer Limits | 19382 MB global, 19847731 KB constant | |----------------'------------------------------------------------------------| | Info: OpenCL C code successfully compiled. | |-----------------.-----------------------------------------------------------| | Grid Resolution | 256 x 256 x 256 = 16777216 | | LBM Type | D3Q19 SRT (FP32/FP32) | | Memory Usage | CPU 272 MB, GPU 1488 MB | | Max Alloc Size | 1216 MB | | Time Steps | 10 | | Kin. Viscosity | 1.00000000 | | Relaxation Time | 3.50000000 | | Reynolds Number | Re < 148 | |---------.-------'-----.-----------.-------------------.---------------------| | MLUPs | Bandwidth | Steps/s | Current Step | Time Remaining | | 246 | 38 GB/s | 15 | 9999 90% | 0s | |---------'-------------'-----------'-------------------'---------------------| | Info: Peak MLUPs/s = 263 |
|----------------.------------------------------------------------------------| | Device ID 0 | gfx90c | | Device ID 1 | gfx90c | | Device ID 2 | gfx90c | |----------------'------------------------------------------------------------| |----------------.------------------------------------------------------------| | Device ID | 0 | | Device Name | gfx90c | | Device Vendor | Advanced Micro Devices, Inc. | | Device Driver | 3380.6 (PAL,HSAIL) | | OpenCL Version | OpenCL C 2.0 | | Compute Units | 8 at 2100 MHz (512 cores, 2.150 TFLOPs/s) | | Memory, Cache | 26899 MB, 16 KB global / 32 KB local | | Buffer Limits | 19382 MB global, 19847731 KB constant | |----------------'------------------------------------------------------------| | Info: OpenCL C code successfully compiled. | |-----------------.-----------------------------------------------------------| | Grid Resolution | 256 x 256 x 256 = 16777216 | | LBM Type | D3Q19 SRT (FP32/FP16S) | | Memory Usage | CPU 272 MB, GPU 880 MB | | Max Alloc Size | 608 MB | | Time Steps | 10 | | Kin. Viscosity | 1.00000000 | | Relaxation Time | 3.50000000 | | Reynolds Number | Re < 148 | |---------.-------'-----.-----------.-------------------.---------------------| | MLUPs | Bandwidth | Steps/s | Current Step | Time Remaining | | 505 | 39 GB/s | 30 | 9998 80% | 0s | |---------'-------------'-----------'-------------------'---------------------| | Info: Peak MLUPs/s = 511 |
|----------------.------------------------------------------------------------| | Device ID 0 | gfx90c | | Device ID 1 | gfx90c | | Device ID 2 | gfx90c | |----------------'------------------------------------------------------------| |----------------.------------------------------------------------------------| | Device ID | 0 | | Device Name | gfx90c | | Device Vendor | Advanced Micro Devices, Inc. | | Device Driver | 3380.6 (PAL,HSAIL) | | OpenCL Version | OpenCL C 2.0 | | Compute Units | 8 at 2100 MHz (512 cores, 2.150 TFLOPs/s) | | Memory, Cache | 26899 MB, 16 KB global / 32 KB local | | Buffer Limits | 19382 MB global, 19847731 KB constant | |----------------'------------------------------------------------------------| | Info: OpenCL C code successfully compiled. | |-----------------.-----------------------------------------------------------| | Grid Resolution | 256 x 256 x 256 = 16777216 | | LBM Type | D3Q19 SRT (FP32/FP16C) | | Memory Usage | CPU 272 MB, GPU 880 MB | | Max Alloc Size | 608 MB | | Time Steps | 10 | | Kin. Viscosity | 1.00000000 | | Relaxation Time | 3.50000000 | | Reynolds Number | Re < 148 | |---------.-------'-----.-----------.-------------------.---------------------| | MLUPs | Bandwidth | Steps/s | Current Step | Time Remaining | | 466 | 36 GB/s | 28 | 9998 80% | 0s | |---------'-------------'-----------'-------------------'---------------------| | Info: Peak MLUPs/s = 501 |
Is it possible to add ready-to-run benchmark for MacOS so we can get more result on Mac? Especially the test is bandwidth limited and Apple silicon should be good at this. Not to mention relatively cheap 64GB+ VRAM as they share the same main memory.
RTX3060 Laptop GPU with 12700H on ASUS ROG M16 Turbo mode (120W GPU TDP) and external laptop fan PS C:\Software\FluidX3D> .\FluidX3D-Benchmark-FP32-FP32-Windows.exe .-----------------------------------------------------------------------------. | ______________ ______________ | | \ ________ | | ________ / | | \ \ | | | | / / | | \ \ | | | | / / | | \ \ | | | | / / | | \ _.-" | | "-._/ / | | \ .-" _ "-. / | | .-" .-" "-. "-./ | | .-" .-"-. "-. | | \ v" "v / | | \ \ / / | | \ \ / / | | \ \ / / | | \ ' / | | \ / | | \ / | | ' ╕ Moritz Lehmann | |----------------.------------------------------------------------------------| | Device ID 0 | NVIDIA GeForce RTX 3060 Laptop GPU | | Device ID 1 | Intel(R) Iris(R) Xe Graphics | |----------------'------------------------------------------------------------| |----------------.------------------------------------------------------------| | Device ID | 0 | | Device Name | NVIDIA GeForce RTX 3060 Laptop GPU | | Device Vendor | NVIDIA Corporation | | Device Driver | 512.78 | | OpenCL Version | OpenCL C 1.2 | | Compute Units | 30 at 1425 MHz (3840 cores, 10.944 TFLOPs/s) | | Memory, Cache | 6143 MB, 840 KB global / 48 KB local | | Buffer Limits | 1535 MB global, 64 KB constant | |----------------'------------------------------------------------------------| | Info: OpenCL C code successfully compiled. | |-----------------.-----------------------------------------------------------| | Grid Resolution | 256 x 256 x 256 = 16777216 | | LBM Type | D3Q19 SRT (FP32/FP32) | | Memory Usage | CPU 272 MB, GPU 1488 MB | | Max Alloc Size | 1216 MB | | Time Steps | 10 | | Kin. Viscosity | 1.00000000 | | Relaxation Time | 3.50000000 | | Reynolds Number | Re < 148 | |---------.-------'-----.-----------.-------------------.---------------------| | MLUPs | Bandwidth | Steps/s | Current Step | Time Remaining | | 2014 | 308 GB/s | 120 | 9999 90% | 0s | |---------'-------------'-----------'-------------------'---------------------| | Info: Peak MLUPs/s = 2019 |
PS C:\Software\FluidX3D> .\FluidX3D-Benchmark-FP32-FP16C-Windows.exe .-----------------------------------------------------------------------------. | ______________ ______________ | | \ ________ | | ________ / | | \ \ | | | | / / | | \ \ | | | | / / | | \ \ | | | | / / | | \ _.-" | | "-._/ / | | \ .-" _ "-. / | | .-" .-" "-. "-./ | | .-" .-"-. "-. | | \ v" "v / | | \ \ / / | | \ \ / / | | \ \ / / | | \ ' / | | \ / | | \ / | | ' ╕ Moritz Lehmann | |----------------.------------------------------------------------------------| | Device ID 0 | NVIDIA GeForce RTX 3060 Laptop GPU | | Device ID 1 | Intel(R) Iris(R) Xe Graphics | |----------------'------------------------------------------------------------| |----------------.------------------------------------------------------------| | Device ID | 0 | | Device Name | NVIDIA GeForce RTX 3060 Laptop GPU | | Device Vendor | NVIDIA Corporation | | Device Driver | 512.78 | | OpenCL Version | OpenCL C 1.2 | | Compute Units | 30 at 1425 MHz (3840 cores, 10.944 TFLOPs/s) | | Memory, Cache | 6143 MB, 840 KB global / 48 KB local | | Buffer Limits | 1535 MB global, 64 KB constant | |----------------'------------------------------------------------------------| | Info: OpenCL C code successfully compiled. | |-----------------.-----------------------------------------------------------| | Grid Resolution | 256 x 256 x 256 = 16777216 | | LBM Type | D3Q19 SRT (FP32/FP16C) | | Memory Usage | CPU 272 MB, GPU 880 MB | | Max Alloc Size | 608 MB | | Time Steps | 10 | | Kin. Viscosity | 1.00000000 | | Relaxation Time | 3.50000000 | | Reynolds Number | Re < 148 | |---------.-------'-----.-----------.-------------------.---------------------| | MLUPs | Bandwidth | Steps/s | Current Step | Time Remaining | | 3523 | 271 GB/s | 210 | 9996 60% | 0s | |---------'-------------'-----------'-------------------'---------------------| | Info: Peak MLUPs/s = 3572 |
PS C:\Software\FluidX3D> .\FluidX3D-Benchmark-FP32-FP16S-Windows.exe .-----------------------------------------------------------------------------. | ______________ ______________ | | \ ________ | | ________ / | | \ \ | | | | / / | | \ \ | | | | / / | | \ \ | | | | / / | | \ _.-" | | "-._/ / | | \ .-" _ "-. / | | .-" .-" "-. "-./ | | .-" .-"-. "-. | | \ v" "v / | | \ \ / / | | \ \ / / | | \ \ / / | | \ ' / | | \ / | | \ / | | ' ╕ Moritz Lehmann | |----------------.------------------------------------------------------------| | Device ID 0 | NVIDIA GeForce RTX 3060 Laptop GPU | | Device ID 1 | Intel(R) Iris(R) Xe Graphics | |----------------'------------------------------------------------------------| |----------------.------------------------------------------------------------| | Device ID | 0 | | Device Name | NVIDIA GeForce RTX 3060 Laptop GPU | | Device Vendor | NVIDIA Corporation | | Device Driver | 512.78 | | OpenCL Version | OpenCL C 1.2 | | Compute Units | 30 at 1425 MHz (3840 cores, 10.944 TFLOPs/s) | | Memory, Cache | 6143 MB, 840 KB global / 48 KB local | | Buffer Limits | 1535 MB global, 64 KB constant | |----------------'------------------------------------------------------------| | Info: OpenCL C code successfully compiled. | |-----------------.-----------------------------------------------------------| | Grid Resolution | 256 x 256 x 256 = 16777216 | | LBM Type | D3Q19 SRT (FP32/FP16S) | | Memory Usage | CPU 272 MB, GPU 880 MB | | Max Alloc Size | 608 MB | | Time Steps | 10 | | Kin. Viscosity | 1.00000000 | | Relaxation Time | 3.50000000 | | Reynolds Number | Re < 148 | |---------.-------'-----.-----------.-------------------.---------------------| | MLUPs | Bandwidth | Steps/s | Current Step | Time Remaining | | 3991 | 307 GB/s | 238 | 9989 90% | 0s | |---------'-------------'-----------'-------------------'---------------------| | Info: Peak MLUPs/s = 4012 |
PS C:\Software\FluidX3D>
@HAL9000COM thanks for the Vega 8 benchmarks! Quick question: Is your RAM is 2x16GB DDR4-3200MT/s? And do you have an idea why the GPU shows up 3 times?
Is it possible to add ready-to-run benchmark for MacOS so we can get more result on Mac? Especially the test is bandwidth limited and Apple silicon should be good at this. Not to mention relatively cheap 64GB+ VRAM as they share the same main memory.
@edmond1992 unfortunately I don't have a Mac, so I can't compile add the executables for MacOS. But the code should work as-is; just compile it as-is with the third line in make.sh and you'll get the FP32 benchmark. Uncomment FP16S/FP16C in src/defines.hpp and recompile to get the other 2 benchmarks.
Cross compile?
Sent from my iPhone
On 23 Oct 2022, at 16:06, Moritz Lehmann @.***> wrote:
Is it possible to add ready-to-run benchmark for MacOS so we can get more result on Mac? Especially the test is bandwidth limited and Apple silicon should be good at this. Not to mention relatively cheap 64GB+ VRAM as they share the same main memory.
@edmond1992https://github.com/edmond1992 unfortunately I don't have a Mac, so I can't compile add the executables for MacOS. But the code should work as-is; just compile it as-is with the third line in make.sh and you'll get the FP32 benchmark. Uncomment FP16S/FP16C in src/defines.hpp and recompile to get the other 2 benchmarks.
— Reply to this email directly, view it on GitHubhttps://github.com/ProjectPhysX/FluidX3D/issues/8#issuecomment-1288045906, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALNCZ732JE6QCL7RKB37QG3WETWXJANCNFSM6AAAAAAQTGZVRY. You are receiving this because you were mentioned.Message ID: @.***>
[https://www.polyu.edu.hk/emaildisclaimer/85A-PolyU_Email_Signature.jpg]
Disclaimer:
This message (including any attachments) contains confidential information intended for a specific individual and purpose. If you are not the intended recipient, you should delete this message and notify the sender and The Hong Kong Polytechnic University (the University) immediately. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited and may be unlawful.
The University specifically denies any responsibility for the accuracy or quality of information obtained through University E-mail Facilities. Any views and opinions expressed are only those of the author(s) and do not necessarily represent those of the University and the University accepts no liability whatsoever for any losses or damages incurred or caused to any party as a result of the use of such information.
@HAL9000COM thanks for the Vega 8 benchmarks! Quick question: Is your RAM is 2x16GB DDR4-3200MT/s? And do you have an idea why the GPU shows up 3 times?
2x32GB DDR4-3200 OC to 3533. No idea why GPU shows up multiple times. After some reboot, it now shows up as two devices.
GTX 1050 on an old gaming laptop. It's amazing I figured out how to even run this and get a benchmark. Now I'm going to try and figure out how to run the simulation on an stl (or similar) file. I know how to use Blender quite well, but this is my first time with visial studio or command line stuff. I'm so out of my depth here 😟
@ProjectPhysX have now added the FP16 benchmarks
RTX 3080 Ti
Updated FP32 (was concurrently baking a fluid in Blender when I ran the last one):
FP16S:
FP16C:
Hi @SirWixy, thank you so much for the benchmarks! Can you post the FP16S and FP16C results too?
Quadro RTX 4000 below. I also tried two Xeon Gold 5218 (2x16 cores), with the FP32/FP32 benchmark they top out at 126MLUPs/s, 20GB/s and 8 steps/s. I did not have the patience to run it to the end. The speedup with GPUs is really dramatic, damn.
@gittigittibangbang thanks for the benchmarks! For the CPU you can just stop it with Ctrl+C after it has leveled at constant performance, and take the last MLUPs/s reading. Can you post the program header with the Xeon Gold for the specs, and performance values for FP16S and FP16C too for the Xeon? Thanks!
|----------------.------------------------------------------------------------| | Device ID 0 | Quadro RTX 4000 | | Device ID 1 | Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz | |----------------'------------------------------------------------------------| |----------------.------------------------------------------------------------| | Device ID | 1 | | Device Name | Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz | | Device Vendor | Intel(R) Corporation | | Device Driver | 6.4.0.37 | | OpenCL Version | OpenCL C 2.0 | | Compute Units | 32 at 2300 MHz (16 cores, 1.178 TFLOPs/s) | | Memory, Cache | 261766 MB, 256 KB global / 32 KB local | | Buffer Limits | 65441 MB global, 128 KB constant
FP32/FP32: 132MLUPs/s, 20GB/s bandwidth, 8 steps/s FP32/FP16C: 270MLUPs/s, 21GB/s bandwidth, 16 steps/s FP32/FP16S: 135MLUPs/s, 10GB/s bandwidth, 8 steps/s
@gittigittibangbang last question: What is the RAM configuration on the Xeon Gold 5218? 8x 32GB DDR4 2667MT/s in quad-channel?
Yes, 8x32GB DDR4 at 2667MHz, but apparently only dual channel according to CPUZ. It seems there's something amiss in the UEFI settings, it should be quad channel.