FFTW.jl icon indicating copy to clipboard operation
FFTW.jl copied to clipboard
verified publisher icon JuliaMath

Unresolved calls when compiling with juliac and --trim

Open Yuan-Ru-Lin opened this issue 4 months ago • 11 comments

The code snippet below (from #308) can't be compiled by juliac with --trim anymore

using FFTW

Base.@ccallable function main()::Cint
    v = fft([0; 1; 2; 1])
    for elem in v
        println(Core.stdout, elem)
    end
    return 0
end

julialauncher +1.12 --project=. --startup-file=no $CFSUSER/.juliaup/juliaup/julia-1.12.0-rc1+0.x64.linux.gnu/share/julia/juliac/juliac.jl --output-exe a.out --compile-ccallable --experimental --trim script2.jl
Verifier error #1: unresolved call from statement FFTW.unsafe_destroy_plan(φ ()::FFTW.FFTWPlan)::Any
Stacktrace:
 [1] foreach(f::typeof(FFTW.unsafe_destroy_plan), itr::Vector{FFTW.FFTWPlan})
   @ Base abstractarray.jl:3188 [inlined]
 [2] destroy_deferred()
   @ FFTW /pscratch/sd/y/yuanru/.julia/packages/FFTW/r6EbH/src/fft.jl:339
 [3] (FFTW.cFFTWPlan{ComplexF64, -1, false, 1})(X::Vector{ComplexF64}, Y::FFTW.FakeArray{ComplexF64, 1}, region::UnitRange{Int64}, flags::UInt32, timelimit::Float64)
   @ FFTW /pscratch/sd/y/yuanru/.julia/packages/FFTW/r6EbH/src/FFTW.jl:74
 [4] plan_fft(X::Vector{ComplexF64}, region::UnitRange{Int64}; flags::UInt32, timelimit::Float64, num_threads::Nothing)
   @ FFTW /pscratch/sd/y/yuanru/.julia/packages/FFTW/r6EbH/src/fft.jl:787 [inlined]
 [5] plan_fft(X::Vector{ComplexF64}, region::UnitRange{Int64})
   @ FFTW /pscratch/sd/y/yuanru/.julia/packages/FFTW/r6EbH/src/fft.jl:777 [inlined]
 [6] fft(x::Vector{ComplexF64}, region::UnitRange{Int64})
   @ AbstractFFTs /pscratch/sd/y/yuanru/.julia/packages/AbstractFFTs/4iQz5/src/definitions.jl:67 [inlined]
 [7] fft(x::Vector{Int64}, region::UnitRange{Int64})
   @ AbstractFFTs /pscratch/sd/y/yuanru/.julia/packages/AbstractFFTs/4iQz5/src/definitions.jl:214
 [8] fft(x::Vector{Int64})
   @ AbstractFFTs /pscratch/sd/y/yuanru/.julia/packages/AbstractFFTs/4iQz5/src/definitions.jl:66 [inlined]
 [9] main()
   @ Main /global/u2/y/yuanru/trimtest/script2.jl:4

Verifier error #2: unresolved call from statement FFTW.unsafe_destroy_plan(φ ()::FFTW.FFTWPlan)::Any
Stacktrace:
 [1] foreach(f::typeof(FFTW.unsafe_destroy_plan), itr::Vector{FFTW.FFTWPlan})
   @ Base abstractarray.jl:3188 [inlined]
 [2] destroy_deferred()
   @ FFTW /pscratch/sd/y/yuanru/.julia/packages/FFTW/r6EbH/src/fft.jl:339
 [3] (FFTW.cFFTWPlan{ComplexF64, -1, false, 1})(X::Vector{ComplexF64}, Y::FFTW.FakeArray{ComplexF64, 1}, region::UnitRange{Int64}, flags::UInt32, timelimit::Float64)
   @ FFTW /pscratch/sd/y/yuanru/.julia/packages/FFTW/r6EbH/src/FFTW.jl:74
 [4] plan_fft(X::Vector{ComplexF64}, region::UnitRange{Int64}; flags::UInt32, timelimit::Float64, num_threads::Nothing)
   @ FFTW /pscratch/sd/y/yuanru/.julia/packages/FFTW/r6EbH/src/fft.jl:787 [inlined]
 [5] plan_fft(X::Vector{ComplexF64}, region::UnitRange{Int64})
   @ FFTW /pscratch/sd/y/yuanru/.julia/packages/FFTW/r6EbH/src/fft.jl:777 [inlined]
 [6] fft(x::Vector{ComplexF64}, region::UnitRange{Int64})
   @ AbstractFFTs /pscratch/sd/y/yuanru/.julia/packages/AbstractFFTs/4iQz5/src/definitions.jl:67 [inlined]
 [7] fft(x::Vector{Int64}, region::UnitRange{Int64})
   @ AbstractFFTs /pscratch/sd/y/yuanru/.julia/packages/AbstractFFTs/4iQz5/src/definitions.jl:214
 [8] fft(x::Vector{Int64})
   @ AbstractFFTs /pscratch/sd/y/yuanru/.julia/packages/AbstractFFTs/4iQz5/src/definitions.jl:66 [inlined]
 [9] main()
   @ Main /global/u2/y/yuanru/trimtest/script2.jl:4

Trim verify finished with 2 errors, 0 warnings.

Failed to compile script2.jl

Here's result of versioninfo()

Julia Version 1.12.0-rc1
Commit 228edd6610b (2025-07-12 20:11 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 256 × AMD EPYC 7713 64-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, znver3)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 256 virtual cores)
Environment:
  JULIA_PROJECT = @work
  JULIA_DEPOT_PATH = /pscratch/sd/y/yuanru/.julia

Yuan-Ru-Lin avatar Aug 07 '25 00:08 Yuan-Ru-Lin

At this point is would be safer (and frankly shorter) to rewrite https://github.com/JuliaMath/FFTW.jl/blob/dbcb5d210be95a0edac920646884784da87f7cab/src/fft.jl#L338-L343 to use a loop rather than foreach. Want to give it a try, @Yuan-Ru-Lin?

timholy avatar Aug 07 '25 08:08 timholy

Changing the code block to

for plan in deferred_destroy_plans
    unsafe_destroy_plan(plan)
end

doesn't work.

I added a type hint to the result of unsafe_destroy_plan because it's a ccall whose result is of type Cvoid; that is,

for plan in deferred_destroy_plans
    unsafe_destroy_plan(plan)::Nothing
end

but juliac still think there's a resolved call

Verifier error #1: unresolved call from statement FFTW.unsafe_destroy_plan(φ ()::FFTW.FFTWPlan)::Nothing

Yuan-Ru-Lin avatar Aug 07 '25 20:08 Yuan-Ru-Lin

OK. I guess the problem is

julia> Base.unwrap_unionall(FFTW.FFTWPlan)
FFTW.FFTWPlan{T<:Union{Float32, Float64, ComplexF64, ComplexF32}, K, inplace}

and something must have changed to make juliac not make it's way through despite the @nospecialized in unsafe_destroy_plan.

I'd recommend filing this as a Julia issue.

timholy avatar Aug 07 '25 21:08 timholy

is that just union splitting giving up?

Moelf avatar Aug 07 '25 23:08 Moelf

The K and inplace are open-ended, so there's no union to split

timholy avatar Aug 08 '25 07:08 timholy

I couldn't find a linked issue. @Yuan-Ru-Lin have you filed one? FFTW is quite essential to us, so we are interested in it being trimable.

laborg avatar Sep 18 '25 09:09 laborg

I've filed an issue on Julia repo. In the meanwhile I've tried to make unsafe_destroy_plan type-stable. I don't know if this breaks something, so suggestions are highly appreciated and there may be a better way to do it for sure.

module FFTWOverrides

using FFTW

const deferred_destroy_plans_f64 = FFTW.FFTWPlan{Float64}[]
const deferred_destroy_plans_c64 = FFTW.FFTWPlan{ComplexF64}[]
const deferred_destroy_plans_f32 = FFTW.FFTWPlan{Float32}[]
const deferred_destroy_plans_c32 = FFTW.FFTWPlan{ComplexF32}[]

@inline function FFTW.unsafe_destroy_plan(plan::FFTW.FFTWPlan{T}) where T
    if T<:Float64 || T<:ComplexF64
        ccall((:fftw_destroy_plan,FFTW.libfftw3), Cvoid, (FFTW.PlanPtr,), plan)
    elseif T<:Float32 || T<:ComplexF32
        ccall((:fftwf_destroy_plan,FFTW.libfftw3f), Cvoid, (FFTW.PlanPtr,), plan)
    end
end

function trylock_destroy_deferred_plans(deferred_destroy_plans)
    if !isempty(deferred_destroy_plans) && trylock(FFTW.fftwlock)
        try
            @inline foreach(FFTW.unsafe_destroy_plan, deferred_destroy_plans)
            empty!(deferred_destroy_plans)
        finally
            unlock(FFTW.fftwlock)
        end
    end
end

function FFTW.destroy_deferred()
    lock(FFTW.deferred_destroy_lock)
    try
        trylock_destroy_deferred_plans(deferred_destroy_plans_f64)
        trylock_destroy_deferred_plans(deferred_destroy_plans_c64)
        trylock_destroy_deferred_plans(deferred_destroy_plans_f32)
        trylock_destroy_deferred_plans(deferred_destroy_plans_c32)
    finally
        unlock(FFTW.deferred_destroy_lock)
    end
end

push_plan(plan::FFTW.FFTWPlan{T}) where T = begin
    if T<:Float64 
        push!(deferred_destroy_plans_f64, plan)
    elseif T<:ComplexF64
        push!(deferred_destroy_plans_c64, plan)
    elseif T<:Float32 
        push!(deferred_destroy_plans_f32, plan)
    elseif T<:ComplexF32
        push!(deferred_destroy_plans_c32, plan)
    end
end

function FFTW.maybe_destroy_plan(plan::FFTW.FFTWPlan)
    while !trylock(FFTW.deferred_destroy_lock)
        GC.safepoint()
    end
    try
        if trylock(FFTW.fftwlock)
            try
                FFTW.unsafe_destroy_plan(plan)
            finally
                unlock(FFTW.fftwlock)
            end
        else
            push_plan(plan)
        end
    finally
        unlock(FFTW.deferred_destroy_lock)
    end
end

end

It's almost a copy-paste. The main edit is the presence of specialized branches for each type Float64, ComplexF64, Float32 and ComplexF32 in the functions: unsafe_destroy_plan and push_plan, in order to manage the corresponding arrays deferred_destroy_plans_*.

In this way the build of the code snippet of #308 with --trim option completes successfully. However, executing the standalone .exe fails with:

fatal: error thrown and no exception handler available.
Core.TypeError(func=:ccall, context="", expected=Symbol, got=FFTW.FakeLazyLibrary(reallibrary=:libfftw3_no_init, on_load_callback=FFTW.var"#fftw_init_check"(), h=Core.Ptr{Core.Nothing}(0x0000000000000000)))

To give more details, the same error is thrown without FFTWOverrides module using --trim=unsafe-warn to pass juliac compilation.

arch-dev avatar Oct 15 '25 09:10 arch-dev

I made a similar attempt at making the destruction stable. Since the only thing we need to destroy a plan is the pointer and deciding library to call (single or double), we could save just this information instead of the whole plan

struct PlanDestructor
    ptr::PlanPtr
    issingle::Bool
end
PlanDestructor(plan::FFTWPlan{<:fftwSingle}) = PlanDestructor(plan.plan, true)
PlanDestructor(plan::FFTWPlan{<:fftwDouble}) = PlanDestructor(plan.plan, false)

Base.convert(::Type{PlanDestructor}, plan::FFTWPlan) = PlanDestructor(plan) # necessary for push! in maybe_destroy_plan

# Replace unsafe_destroy_plan with this
unsafe_destroy_plan(plan::FFTWPlan) = unsafe_destroy_plan(PlanDestructor(plan))
function unsafe_destroy_plan(dstr::PlanDestructor)
    if dstr.issingle
        ccall((:fftwf_destroy_plan,libfftw3f), Cvoid, (PlanPtr,), dstr.ptr)
    else
        ccall((:fftw_destroy_plan,libfftw3), Cvoid, (PlanPtr,), dstr.ptr)
    end
end

# Replace deferred_destroy_plans with this
const deferred_destroy_plans = PlanDestructor[]

This passes the FFTW tests and juliac build, but gives the same error as you describe upon execution.

johroj avatar Oct 20 '25 09:10 johroj

Having a special PlanDestructor type, to make deferred_destroy_plans a concretely typed container, seems like a good solution here.

stevengj avatar Oct 21 '25 23:10 stevengj

I guess this could be merged right away, but the issingle field does not feel like the nicest option. I tried using the library itself, but FakeLazyLibrary is not concrete. The comment at https://github.com/JuliaMath/FFTW.jl/blob/master/src/FFTW.jl#L33 indicates that this should be changed anyway, and we still wont see static compilation until then, so maybe the best option is to just hold on?

johroj avatar Oct 22 '25 12:10 johroj

I made another attempt and managed to bulid and run a scrip with FFTW this time. Most things can probably be merged into FFTW, but the user also needs to run some extra code before calling e.g. fft. See https://github.com/JuliaMath/FFTW.jl/pull/327 for details.

johroj avatar Nov 15 '25 14:11 johroj