comparison with openpiv

[alexlib]

Dear QuickPIV authors, @Marc-3d

Thanks for writing another open-source PIV software and citing our effort with OpenPIV. There are a few claims in the paper that we'd like to understand better and if possible, to improve our software based on your findings:

1. "To our knowledge, quickPIV is the only free PIV software that offers normalized squared error cross-correlation (NSQECC)" - as far as we understand, you probably used an older version of OpenPIV Python - since we have the normalized and non-normalized correlation functions for a long time (we're now on 0.23.8 and it's there since 0.22.4 or so)
2. The speed of QuickPIV - we do not quite understand what is the benchmark and conditions under which you compare the three software packages: QuickPIV, OpenPIV Python, and OpenPIV C++. The C++ version is not very actively maintained and we are not sure which version you compiled, on what compiler, and with which flags. As far as we know, Julia is a fast numerical language, but it cannot be too fast compared to C++. Therefore, our understanding is that the comparison can be subjective.

Thanks in advance, Alex Liberzon, one of the OpenPIV developers @alexlib

[Marc-3d]

Dear Alex Liberzon,

First of all, thanks of developing and maintaining the openPIV platform. We started developing our software some two years ago, when the openPIV Python didn't yet offer 3D analyses. From the beginning we used openPIV as a reference for our software.

Answering your points:

1-. As far as I know, openPIV Python normalizes cross-correlation by subtracting the mean from the interrogation/search areas. QuickPIV implements ZNCC, which is basically mean subtraction and division by the standard deviation, but this was not enough to provide good results on actin-labelled embryonic data. We then implemented NSQECC, where each value in the cross-correlation matrix is the sum of squared differences between the pixels of the interrogation and search areas. From our knowledge of other PIV software, this feature is unique to quickPIV.

2-. You can find the performance evaluation in the "Evaluation" branch. Here are some notes about the speed results:

  Comparison of FFT cross-correlation performance:

     As I mention in "openPIV_python_performance.ipynb", we copy-pasted the cross-correlation code from process.pyx 
     in v0.22.3, and we took the minimum (fastest) speed from 1000 repeats. I believe that the only difference in further 
     versions of openPIV Python is that process.pyx introduces a rounding of the size of the inputs to the closest power of 2. 
     We excluded this heuristic from the evaluation. The last cell in "openPIV_python_performance.ipynb" shows the evaluation
     of v0.23.4 FFT cross-correlation function, where we observe that this rounding seems to improve performance on average, 
     but in some cases it leads to slower FFT cross-correlation. 

     The code to evaluate FFT cross-correlation performance is included in "openPIV_cpp_script.cpp", and it was compiled 
     with "g++ openPIV_cpp_script.cpp -lfftw3".  

 Comparison of complete PIV analyses: 

     The evaluation of openPIV Python is included in the notebook, and is quite straightforward. For the evaluation of 
     openPIV C++ I had to hack into the application. After excluding loading the inputs from the performance measurements, I  
     believe I obtained faithful measurements of openPIV C++ 2D PIV analyses. 

     Since FFT cross-correlation is equally fast between Julia and C++ (both call FFTW), the slight speed gain of quickPIV must
     be explained by the design of quickPIV or  by "under the hood" Julia compile-time optimizations. I can only speak good
     about Julia, which combines compile-time optimizations + high-level environment. Overall, Julia allowed us to easily
     structure our code around in-place operations to reduce memory allocations, and to exploit SIMD operations. Namely, 
     we used SIMD to optimize the performance of the "findmax" function, to locate the maximum peak in the 
     cross-correlation matrix. 

I hope I could answer your questions, and I am looking forward to speaking further with the openPIV community. Marc Pereyra

[alexlib]

Thanks a lot Marc @Marc-3d for the detailed answer. i suggest to join forces to create a repo that provides some kind of Python/Julia/C++ notebooks that use the same data and run in mybinder or another service to get some sort of benchmarking. This is rather useful for both communities.

I'm surprised that the sum of squared differences perform (in terms of PIV quality) better than FFT-based algorithm, though I've seen it in the past. What puzzles here is that one can rewrite both in terms of FFT and to show that the number of operations is actually larger for the SSD (sum-squared-differences). So I'd need to learn something from your code. I agree that Julia is a great language and has its merit and probably would be Python's replacement in some cases. Is it easy to install and how can it run on the cloud? We'll also "borrow" from your code the use of VTK - this is something we dream for a long time and never had time to implement.

Best luck with the quickPIV, Alex

[Marc-3d]

Establishing a common benchmark sounds great. I will learn about mybinder, and after Christmas we can start putting together the on-line benchmark.

Julia is easy to install locally. I haven't tried MacOS, but installing Julia should be as easy as downloading and uncompressing the desired prebuild julia version, and adding the directory of the Julia executable to PATH. I have no experience with running code on the cloud, but I am aware that "JuliaHub" exists, which is a paid service.

I am happy to read that the VTK functions are useful to you. The code for exporting VTK files should be easy to translate into Python. If you have any questions or suggestions feel free to contact me.

Best regards, Marc

[ErichZimmer]

Hi all :D, Does quickPIV support parallel processing? I plan on doing an extensive test with 18,000 synthetically generated images with dimensions of 256x256px (except the spatial test which goes up to 30MP). Parallel processing would be very useful and won't leave quickPIV in the dust :)

Regards, Erich

[Marc-3d]

Hi Erich,

It does not support it. I will have to read into it and try to add it on time for your tests!

In my TODO list is also to add support for multithreaded PIV analyses, which wasn't possible some time ago because of shared FFT plans between threads. But it think FFTW.jl now supports multithreaded FFT's.

Best, Marc

[ErichZimmer]

No worries, there is still plenty of time. :)

[ErichZimmer]

@Marc-3d , Using Dierckx.jl, the OpenPIV-Python windef script can be incorporated into Julia for a very fast and accurate window deformation script. Simply shifting the interrogation windows leaves some room for improvements on multi-pass evaluations of non-linear flows. Maybe this is something to look into?