Trixi.jl icon indicating copy to clipboard operation
Trixi.jl copied to clipboard

WIP: Switch to P4est with CBinding v1

Open sloede opened this issue 2 years ago • 7 comments

This PR prepares Trixi for working with trixi-framework/P4est.jl#42 or trixi-framework/P4est.jl#43. To use it, you need to manually dev a local clone of said PRs, e.g., from within a clone of this Trixi branch, execute

julia --project -e 'using Pkg; Pkg.develop("path/to/P4est.jl/clone/on/branch/cbinding-v1")'

sloede avatar Sep 01 '21 04:09 sloede

When running

julia> using Revise, Trixi

julia> trixi_include("examples/p4est_2d_dgsem/elixir_advection_basic.jl")

with the current version (and the appropriate dev'd P4est.jl branch), I get the following error:

ERROR: LoadError: MethodError: no method matching unsafe_convert(::Type{CBinding.Cptr{CBinding.Cconst{Int32}}}, ::Matrix{Int32})
Closest candidates are:
  unsafe_convert(::Type{CBinding.Cptr{T}}, ::Vector{T} where T) where T at /home/mschlott/.julia/packages/CBinding/0akLv/src/pointers.jl:40
  unsafe_convert(::Type{CBinding.Cptr{T}}, ::Ptr) where T at /home/mschlott/.julia/packages/CBinding/0akLv/src/pointers.jl:38
  unsafe_convert(::Type{CBinding.Cptr{T}}, ::Ref) where T at /home/mschlott/.julia/packages/CBinding/0akLv/src/pointers.jl:39
  ...
Stacktrace:
  [1] macro expansion
    @ ./boot.jl:0 [inlined]
  [2] funccall(::Type{CBinding.Cptr{P4est.libp4est.var"c\"struct p4est_connectivity\""}}, ::Type{Tuple{Int32, Int32, Int32, CBinding.Cptr{CBinding.Cconst{Float64}}, CBinding.Cptr{CBinding.Cconst{Int32}}, CBinding.Cptr{CBinding.Cconst{Int32}}, CBinding.Cptr{CBinding.Cconst{Int8}}, CBinding.Cptr{CBinding.Cconst{Int32}}, CBinding.Cptr{CBinding.Cconst{Int32}}, CBinding.Cptr{CBinding.Cconst{Int32}}, CBinding.Cptr{CBinding.Cconst{Int8}}}}, ::Val{:cdecl}, ::P4est.libp4est.Cbinding_p4est_connectivity_new_copy{:p4est_connectivity_new_copy}, ::Int64, ::Int64, ::Int64, ::Ptr{Nothing}, ::Ptr{Nothing}, ::Matrix{Int32}, ::Matrix{Int8}, ::Ptr{Nothing}, ::Vector{Int32}, ::Ptr{Nothing}, ::Ptr{Nothing})
    @ CBinding ~/.julia/packages/CBinding/0akLv/src/functions.jl:32
  [3] macro expansion
    @ ~/.julia/packages/CBinding/0akLv/src/functions.jl:65 [inlined]
  [4] funccall(::P4est.libp4est.Cbinding_p4est_connectivity_new_copy{:p4est_connectivity_new_copy}, ::Int64, ::Int64, ::Int64, ::Ptr{Nothing}, ::Ptr{Nothing}, ::Matrix{Int32}, ::Matrix{Int8}, ::Ptr{Nothing}, ::Vector{Int32}, ::Ptr{Nothing}, ::Ptr{Nothing})
    @ CBinding ~/.julia/packages/CBinding/0akLv/src/functions.jl:46
  [5] (::P4est.libp4est.Cbinding_p4est_connectivity_new_copy{:p4est_connectivity_new_copy})(c"num_vertices"::Int64, c"num_trees"::Int64, c"num_corners"::Int64, c"vertices"::Ptr{Nothing}, c"ttv"::Ptr{Nothing}, c"ttt"::Matrix{Int32}, c"ttf"::Matrix{Int8}, c"ttc"::Ptr{Nothing}, c"coff"::Vector{Int32}, c"ctt"::Ptr{Nothing}, c"ctc"::Ptr{Nothing})
    @ P4est.libp4est ~/.julia/packages/CBinding/0akLv/src/context_c.jl:713
  [6] connectivity_structured(n_cells_x::Int64, n_cells_y::Int64, periodicity::Tuple{Bool, Bool})
    @ Trixi /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/meshes/p4est_mesh.jl:436
  [7] P4estMesh(trees_per_dimension::Tuple{Int64, Int64}; polydeg::Int64, mapping::Nothing, faces::Nothing, coordinates_min::Tuple{Float64, Float64}, coordinates_max::Tuple{Float64, Float64}, RealT::Type, initial_refinement_level::Int64, periodicity::Bool, unsaved_changes::Bool)
    @ Trixi /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/meshes/p4est_mesh.jl:166
  [8] top-level scope
    @ /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/examples/p4est_2d_dgsem/elixir_advection_basic.jl:20
  [9] include
    @ ./Base.jl:387 [inlined]
 [10] #trixi_include#829
    @ /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/auxiliary/special_elixirs.jl:33 [inlined]
 [11] trixi_include
    @ /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/auxiliary/special_elixirs.jl:33 [inlined]
 [12] #trixi_include#831
    @ /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/auxiliary/special_elixirs.jl:36 [inlined]
 [13] trixi_include(elixir::String)
    @ Trixi /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/auxiliary/special_elixirs.jl:36
 [14] top-level scope
    @ REPL[2]:1
in expression starting at /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/examples/p4est_2d_dgsem/elixir_advection_basic.jl:20

Any idea what could be wrong or where I might need to poke around, @krrutkow? To be honest, I have no clue where to even start...

sloede avatar Sep 01 '21 04:09 sloede

Good find! The automatic array-to-pointer conversion was only defined on a Vector, not AbstractArray. If you define this, does it work?

Base.unsafe_convert(::Type{Cptr{T}}, x::AbstractArray) where {T} = Core.Intrinsics.bitcast(Cptr{T}, Base.unsafe_convert(Ptr{eltype(x)}, x))

If so, I will add that to the next update to CBinding.

krrutkow avatar Sep 01 '21 12:09 krrutkow

Good find! The automatic array-to-pointer conversion was only defined on a Vector, not AbstractArray. If you define this, does it work?

Base.unsafe_convert(::Type{Cptr{T}}, x::AbstractArray) where {T} = Core.Intrinsics.bitcast(Cptr{T}, Base.unsafe_convert(Ptr{eltype(x)}, x))

If so, I will add that to the next update to CBinding.

This seems to be working, I added it to https://github.com/krrutkow/P4est.jl/tree/cbinding-v1 in https://github.com/krrutkow/P4est.jl/commit/0a95340a58a217adc55f3d9834ab0ca5510d4ce3. Thanks!

I'll continue fixing up Trixi until I run into the next inexplicable issue...

sloede avatar Sep 01 '21 16:09 sloede

OK, so one thing I can already report is that the REPL regularly keeps dying on me: either during errors, or when I rerun a previously failing and now hopefully fixed example, or just when I exit the REPL. The error messages usually look something like this

julia> 
munmap_chunk(): invalid pointer

signal (6): Aborted
in expression starting at none:0
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7f9028c513ed)
unknown function (ip: 0x7f9028c5947b)
unknown function (ip: 0x7f9028c596cb)
p4est_connectivity_destroy at /workspace/srcdir/p4est-2.3.2/src/p4est_connectivity.c:413
macro expansion at /home/mschlott/.julia/packages/CBinding/0akLv/src/functions.jl:0 [inlined]
funccall at /home/mschlott/.julia/packages/CBinding/0akLv/src/functions.jl:32 [inlined]
macro expansion at /home/mschlott/.julia/packages/CBinding/0akLv/src/functions.jl:65 [inlined]
funccall at /home/mschlott/.julia/packages/CBinding/0akLv/src/functions.jl:46 [inlined]
Cbinding_p4est_connectivity_destroy at /home/mschlott/.julia/packages/CBinding/0akLv/src/context_c.jl:713 [inlined]
destroy_mesh at /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/meshes/p4est_mesh.jl:46
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2237 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2419
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1703 [inlined]
run_finalizer at /buildworker/worker/package_linux64/build/src/gc.c:278
jl_gc_run_finalizers_in_list at /buildworker/worker/package_linux64/build/src/gc.c:365
run_finalizers at /buildworker/worker/package_linux64/build/src/gc.c:394 [inlined]
jl_gc_run_all_finalizers at /buildworker/worker/package_linux64/build/src/gc.c:480
jl_atexit_hook at /buildworker/worker/package_linux64/build/src/init.c:240
repl_entrypoint at /buildworker/worker/package_linux64/build/src/jlapi.c:703
main at julia (unknown line)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x4007d8)
Allocations: 95786701 (Pool: 95749172; Big: 37529); GC: 64
Aborted (core dumped)

Any idea where this might come from? Can this related be to https://github.com/analytech-solutions/CBinding.jl/issues/61? Or is this related to some finalizer funny business?

sloede avatar Sep 01 '21 16:09 sloede

OK, another weird issue: Directly using the macro version @P4EST_MAXLEVEL sometimes fails, sometimes it doesn't. What didn't work: https://github.com/trixi-framework/Trixi.jl/blob/6ab71e91d659eb223ca2034dd0fead52f448c4a3/src/solvers/dgsem_p4est/containers_2d.jl#L53-L54 with error message (during using Trixi already!):

no method matching var"@c\"P4EST_MAXLEVEL\""(::LineNumberNode, ::Module, ::Expr)
[ Info: Precompiling Trixi [a7f1ee26-1774-49b1-8366-f1abc58fbfcb]
ERROR: LoadError: LoadError: LoadError: LoadError: LoadError: LoadError: LoadError: MethodError: no method matching var"@c\"P4EST_MAXLEVEL\""(::LineNumberNode, ::Module, ::Expr)
Closest candidates are:
  var"@c\"P4EST_MAXLEVEL\""(::LineNumberNode, ::Module) at /home/mschlott/.julia/packages/CBinding/0akLv/src/context_c.jl:798
Stacktrace:
  [1] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
  [2] include(x::String)
    @ Trixi /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/Trixi.jl:16
  [3] top-level scope
    @ /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/solvers/dgsem_p4est/containers.jl:671
  [4] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
  [5] include(x::String)
    @ Trixi /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/Trixi.jl:16
  [6] top-level scope
    @ /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/solvers/dgsem_p4est/dg.jl:47
  [7] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
  [8] include(x::String)
    @ Trixi /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/Trixi.jl:16
  [9] top-level scope
    @ /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/solvers/dg.jl:478
 [10] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
 [11] include(x::String)
    @ Trixi /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/Trixi.jl:16
 [12] top-level scope
    @ /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/solvers/solvers.jl:8
 [13] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
 [14] include(x::String)
    @ Trixi /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/Trixi.jl:16
 [15] top-level scope
    @ /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/Trixi.jl:95
 [16] include
    @ ./Base.jl:386 [inlined]
 [17] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1213
 [18] top-level scope
    @ none:1
 [19] eval
    @ ./boot.jl:360 [inlined]
 [20] eval(x::Expr)
    @ Base.MainInclude ./client.jl:446
 [21] top-level scope
    @ none:1
in expression starting at /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/solvers/dgsem_p4est/containers_2d.jl:54
in expression starting at /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/solvers/dgsem_p4est/containers_2d.jl:5
in expression starting at /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/solvers/dgsem_p4est/containers.jl:5
in expression starting at /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/solvers/dgsem_p4est/dg.jl:5
in expression starting at /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/solvers/dg.jl:5
in expression starting at /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/solvers/solvers.jl:5
in expression starting at /mnt/ssd/home/mschlott/hackathon/main-Trixi.jl/src/Trixi.jl:1
ERROR: Failed to precompile Trixi [a7f1ee26-1774-49b1-8366-f1abc58fbfcb] to /home/mschlott/.julia/compiled/v1.6/Trixi/jl_AD7AzM.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::Base.TTY, internal_stdout::Base.TTY)
   @ Base ./loading.jl:1360
 [3] compilecache(pkg::Base.PkgId, path::String)
   @ Base ./loading.jl:1306
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1021
 [5] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:914
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:901

julia>

Funnily, the following does work: https://github.com/trixi-framework/Trixi.jl/blob/95894475837ab5db08258b45db8a6caa420247fe/src/solvers/dgsem_p4est/containers_2d.jl#L53-L55

Any idea what's going on here @krrutkow?

sloede avatar Sep 01 '21 16:09 sloede

The macro parsing of Julia is probably grabbing everything after @P4EST_MAXLEVEL so if you wrap it with parentheses it will prevent that from happening. (@P4EST_MAXLEVEL) - l

krrutkow avatar Sep 01 '21 16:09 krrutkow

@sloede Regarding https://github.com/trixi-framework/Trixi.jl/pull/835#issuecomment-910452931, I think the pointer being destroyed mesh.p4est.connectivity is the pointer to the connectivity pointer. You probably need mesh.p4est.connectivity[] to get the pointer you want to destroy.

Currently pointers that are arguments to C functions are automatically converted, but perhaps only conversion to/from void * should be automatic and the rest must be explicitly converted. I suppose being less Julian is actually better in this situation...

krrutkow avatar Sep 02 '21 15:09 krrutkow

Can be closed after merging https://github.com/trixi-framework/Trixi.jl/pull/1184?

ranocha avatar Jan 02 '23 07:01 ranocha

Superseded by https://github.com/trixi-framework/Trixi.jl/pull/1184.

sloede avatar Jan 25 '23 12:01 sloede