silx icon indicating copy to clipboard operation
silx copied to clipboard

Fix FFT norm (pyfftw & numpy)

Open leonroussel opened this issue 3 years ago • 4 comments

Fixing problem on normalization of FFT

Usual normalization is on backward FFT ( ./1 on forward & ./N on backward) but we can expect to have a non-normalize FFT (with parameter normalize="none"), this option is not implemented correctly

Solution & modifications :

  • numpy : norm="backward" on FFT & norm="forward" on IFFT
  • pyfftw : normalise_idft=True on FFT & normalise_idft=False on IFFT

(There is also a problem with pyfftw implementation, it does not have the expected behavior with the last version (0.13), https://github.com/pyFFTW/pyFFTW/issues/341 but the solution normally bypass this problem)

TODO

  • [x] ~~requirements for numpy version ? >= 1.12 ? or different in setup.py ? (if >= 1.10, a case could be avoided in npfft.py)~~
  • [ ] normalization "by hand" if numpy version do not allow normalization="none" ? or keep a warning ?
  • [ ] need to be tested with both version of pyfftw 0.12 & 0.13
  • [x] ~~more efficient solution than np_version in ["1.8", ...]~~

leonroussel avatar Jun 28 '22 08:06 leonroussel

Failure on old numpy implem ? --> "backward" & "forward" keywords only for version after 1.20

================================== FAILURES =================================== ... venv_test\lib\site-packages\numpy\fft_pocketfft.py:367: in rfft if norm is not None and _unitary(norm): ... E ValueError: Invalid norm value backward, should be None or "ortho". venv_test\lib\site-packages\numpy\fft_pocketfft.py:88: ValueError

next commit fixes it

leonroussel avatar Jun 28 '22 11:06 leonroussel

Still CI failures : ================================== FAILURES =================================== _____________________ TestMaskToolsWidget.testWithAnImage _____________________ ...

leonroussel avatar Jun 28 '22 12:06 leonroussel

Thanks!

From what I see in requirements.txt and pyproject.toml, the minimum required numpy version is 1.12, not 1.8.

To simplify version comparison a little bit, you can use from pkg_resources import parse_version, eg. parse_version("1.10.2") > parse_version("1.8.0").

The latest CI failures seem related to GUI (happens from time to time)

pierrepaleo avatar Jun 28 '22 13:06 pierrepaleo

Ok it seems the same for me, so I removed code that take into account version 1.8 & 1.9.

Ho ! from pkg_resources import parse_version was exactly what I was looking for. Thanks

leonroussel avatar Jun 28 '22 13:06 leonroussel

Test fails for numpy == 1.19.5. Seems to be because not all normalization were available for this version.

Just tested on my machine:

  • test_norms fails with numpy==1.19.5
  • test_norms passes with numpy==1.20.0

pierrepaleo avatar Sep 30 '22 12:09 pierrepaleo

Tested with pyfftw 0.12 and 0.13.

pierrepaleo avatar Sep 30 '22 12:09 pierrepaleo