ImageFiltering.jl icon indicating copy to clipboard operation
ImageFiltering.jl copied to clipboard

rework gabor filter and add log gabor filter

Open johnnychen94 opened this issue 3 years ago • 6 comments

The previous Kernel.gabor function has three drawbacks:

  • it eagerly allocates memory, and thus makes optimization quite difficult
  • it returns Tuple{Matrix, Matrix} as real and imaginary part; we should just return Matrix{Complex{Float64}} instead
  • it doesn't have good documentation or examples... It takes quite a while for me to understand the usage of it from other resources...

benchmark on gabor filter:

using ImageFiltering

# new lazy Gabor array
kern = Kernel.Gabor((256, 256), 30, 0)
@btime collect($kern); # 1.157 ms (2 allocations: 1.00 MiB)

# old gabor function
σ, θ, λ, γ, ψ = 10, 0, 30, 0.5, 0
@btime Kernel.gabor(256,256,σ,θ,λ,γ,ψ); # 2.911 ms (51 allocations: 12.60 MiB)

Docs Preview: https://juliaimages.org/ImageFiltering.jl/previews/PR235/demos

TODO:

  • [x] add tests
  • [x] revisit the implementation details
  • [x] add doc for log gabor
  • [x] add function reference entries to documentation

The implementation of the log gabor filter heavily follows https://www.peterkovesi.com/matlabfns/PhaseCongruency/Docs/convexpl.html so I'm wondering if @peterkovesi could help review this.

johnnychen94 avatar Nov 01 '21 22:11 johnnychen94

Codecov Report

Merging #235 (e0f182a) into master (424523c) will increase coverage by 0.30%. The diff coverage is 100.00%.

:exclamation: Current head e0f182a differs from pull request most recent head b1981ea. Consider uploading reports for the commit b1981ea to get more accurate results Impacted file tree graph

@@            Coverage Diff             @@
##           master     #235      +/-   ##
==========================================
+ Coverage   92.14%   92.45%   +0.30%     
==========================================
  Files          12       14       +2     
  Lines        1642     1709      +67     
==========================================
+ Hits         1513     1580      +67     
  Misses        129      129              
Impacted Files Coverage Δ
src/kernel.jl 99.02% <ø> (-0.21%) :arrow_down:
src/gaborkernels.jl 100.00% <100.00%> (ø)
src/kernel_deprecated.jl 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 424523c...b1981ea. Read the comment docs.

codecov[bot] avatar Nov 01 '21 22:11 codecov[bot]

I'm marking this as ready as I don't plan to add any further major changes to this PR. I'm going to add phase congruency filter next, and then finally the FSIM quality index https://github.com/JuliaImages/ImageQualityIndexes.jl/issues/25. I'll keep this PR pending until everything downstream is ready.

ImagePhaseCongruency.jl is really a good reference but it has a strong MATLAB legacy, I believe we can get things faster and give it better composability by properly resolving and designing the function.

johnnychen94 avatar Nov 02 '21 20:11 johnnychen94

Benchmark shows that F32 on LogGabor is somewhat slow. I'll try to investigate it before merging this.

    BenchmarkGroup:
        2-element BenchmarkTools.BenchmarkGroup:
          tags: []
          "Gabor" => 6-element BenchmarkTools.BenchmarkGroup:
                  tags: []
                  "Gabor_ComplexF32_19×19" => Trial(5.625 μs)
                  "Gabor_ComplexF32_256×256" => Trial(994.030 μs)
                  "Gabor_ComplexF32_512×512" => Trial(3.857 ms)
                  "Gabor_ComplexF64_19×19" => Trial(7.870 μs)
                  "Gabor_ComplexF64_256×256" => Trial(1.440 ms)
                  "Gabor_ComplexF64_512×512" => Trial(5.553 ms)
          "LogGabor" => 6-element BenchmarkTools.BenchmarkGroup:
                  tags: []
                  "LogGabor_ComplexF64_19×19" => Trial(13.903 μs)
                  "LogGabor_ComplexF64_256×256" => Trial(2.422 ms)
                  "LogGabor_ComplexF64_512×512" => Trial(9.884 ms)
                  "LogGabor_ComplexF32_19×19" => Trial(68.269 μs)
                  "LogGabor_ComplexF32_256×256" => Trial(11.896 ms)
                  "LogGabor_ComplexF32_512×512" => Trial(51.113 ms)

Writing benchmark isn't something pleasant, but when you get the report, oh that's a surprise!

johnnychen94 avatar Nov 03 '21 10:11 johnnychen94

I am very pleased and impressed that this work is being done. Please help yourself to any code you think is useful in ImagePhaseCongruency I am sure much of it can be considerably improved and made more interoperable with the rest of the Images ecosystem.

Another function from ImagePhaseCongruency that I think is deserving of being included in Images is perfft2(). It is very useful when one wants to perform filtering in the frequency domain with minimal artifacts introduced from the boundary discontinuities. See https://peterkovesi.github.io/ImagePhaseCongruency.jl/dev/examples/#Fourier-transform-of-Moisan's-periodic-image-component-1

Nate that I am now retired and find myself not doing very much coding these days. Indeed at the moment I am often off the grid for extended periods of time. Thus I am not in a position to provide code reviews but should you have any questions about my code I will be more than happy to try to answer them. In the meantime I will be cheering you on from the sidelines!

peterkovesi avatar Nov 03 '21 13:11 peterkovesi

@peterkovesi Thanks for the information! I've made an initial implementation of the phasecong3 and it turns out that I only get 2x slower 😂 because of the repeated computation as I noted in https://github.com/JuliaImages/ImageFiltering.jl/pull/235#discussion_r741936206. The idea of separating the log Gabor frequency filter and the log Gaussian angular filter here to decouple the orientation and scale loops is really really insightful.

Profiling my initial implementation code shows that almost 80% of the computation is spent on the log Gabor filter, which is definitely a performance hotspot.

That said, this PR is far from ready or satisfying.

johnnychen94 avatar Nov 03 '21 22:11 johnnychen94