`getproperty(::Type{Self}, name::Symbol) where Self<:ADTExample` results in huge numbers of invalidations`

[chriselrod]

inserting getproperty(::Type{Self}, name::Symbol) where Self<:ADTExample @ ExamplePackage ~/.julia/packages/Expronicon/Ms5h6/src/adt/emit.jl:271 invalidated:
   backedges:  1: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::Type{<:Tuple{Any, Any}}, ::Symbol) (1 children)
               2: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::Type{<:Tuple{Any, SymbolicUtils.BasicSymbolic{SymbolicUtils.FnType{Tuple{Any, Real}, Vector{Real}}}}}, ::Symbol) (1 children)
               3: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::Type{<:Union{Bool, Float16, Float32, Float64, Int16, Int32, Int64, Int8, UInt16, UInt32, UInt64, UInt8, SIMDTypes.Bit}}, ::Symbol) (1 children)
               4: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::Type{<:Tuple{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolve.NonlinearSolveTag, Float64}, Float64}, Val{2}}}, ::Symbol) (2 children)
               5: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::Type{<:Tuple{ForwardDiff.Dual{ForwardDiff.Tag{NonlinearSolve.NonlinearSolveTag, Float32}, Float32}, Val{2}}}, ::Symbol) (2 children)
               6: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::Type{<:CartesianIndices}, ::Symbol) (3 children)
               7: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::Type{<:ColorTypes.Colorant}, ::Symbol) (3 children)
               8: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::Type{<:Tuple{Vararg{T, _A}}} where {T, _A}, ::Symbol) (4 children)
               9: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::Type{<:Function}, ::Symbol) (18 children)
              10: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::Type{S} where S<:Tuple, ::Symbol) (24 children)
              11: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::Type{T} where T<:SymbolicUtils.Symbolic, ::Symbol) (32 children)
              12: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::DataType, ::Symbol) (3758 children)
              13: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::Type, ::Symbol) (4639 children)
              14: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::Union, ::Symbol) (9836 children)
              15: superseding getproperty(x::Type, f::Symbol) @ Base Base.jl:32 with MethodInstance for getproperty(::UnionAll, ::Symbol) (32538 children)

If using OhMyREPL, the REPL up for several seconds after this. Additionally, some examples of using GPUCompiler and StaticCompiler start failing because these invalidations, emitting a mountain of allocations and apply_generics.

using RecursiveFactorization, StrideArraysCore, Static, LinearAlgebra, StaticCompiler

function _swap_rows!(B::AbstractVector, i::Integer, j::Integer)
    @inbounds B[i], B[j] = B[j], B[i]
    B
end

function _swap_rows!(B::AbstractMatrix, i::Integer, j::Integer)
    @inbounds for col in 1:size(B, 2)
        B[i, col], B[j, col] = B[j, col], B[i, col]
    end
    B
end
function _ipiv_rows!(A::LU, order::OrdinalRange, B::AbstractVecOrMat)
    @inbounds for i in order
        if i != A.ipiv[i]
            _swap_rows!(B, i, A.ipiv[i])
        end
    end
    B
end
_apply_ipiv_rows!(A::LU, B::AbstractVecOrMat) = _ipiv_rows!(A, 1:length(A.ipiv), B)
function _jscldiv!(A::LU, B::AbstractVecOrMat)
    _apply_ipiv_rows!(A, B)
    _jscldiv!(UpperTriangular(A.factors), _jscldiv!(UnitLowerTriangular(A.factors), B))
end
function _jscldiv!(A::UpperTriangular, b::AbstractVector, x::AbstractVector = b)
    n = size(A, 2)
    j = n
    @inbounds while j > 0
        xj = x[j] = A.data[j, j] \ b[j]
        i = j - 1
        while i > 0
            b[i] -= A.data[i, j] * xj
            i -= 1
        end
        j -= 1
    end
    # @inbounds for j in n:-1:1
    #     for i in (j - 1):-1:1
    #         b[i] -= A.data[i, j] * xj
    #     end
    # end
    x
end
function _jscldiv!(A::UnitLowerTriangular, b::AbstractVector, x::AbstractVector = b)
    n = size(A, 2)
    @inbounds for j in 1:n
        xj = x[j] = b[j]
        for i in (j + 1):n
            b[i] -= A.data[i, j] * xj
        end
    end
    x
end

@inline function linsolve!(A::Ptr{T}, b::Ptr{T}, ::Val{N}) where {T, N}
    D = static(N)
    piv = Ref{NTuple{N, Int}}()
    # piv = Buffer{N, Int}()
    GC.@preserve piv begin
        F = RecursiveFactorization.lu!(PtrArray(A, (D, D)),
            PtrArray(Base.unsafe_convert(Ptr{Int},piv), (D,)), check = false)
        _jscldiv!(F, PtrArray(b, (D,)))
    end
    return b
end
struct SizedLinsolve{N} <: Function
end;

(::SizedLinsolve{N})(A::Ptr{T}, b::Ptr{T}) where {N, T} = linsolve!(A, b, Val(N))

N = 4
f = SizedLinsolve{4}();
A = rand(N, N); b = rand(N)
x = A\b; f(pointer(A), pointer(b))
x ≈ b
path = compile_shlib(f, (Ptr{Float64}, Ptr{Float64}), "./", filename="jsclinsolve$N")

@time using OrdinaryDiffEq, ModelingToolkit
@time using Expronicon, RealDot, StructArrays, SparseInverseSubset, ChainRules, ArrayInterface

path = compile_shlib(f, (Ptr{Float64}, Ptr{Float64}), "./", filename="jsclinsolve$N")

using Expronicon.ADT: @adt

@adt Message begin
    Quit
    struct Move
        x::Int
        y::Int
    end

    Write(::String)
    ChangeColor(::Int, ::Int, ::Int)
end
path = compile_shlib(f, (Ptr{Float64}, Ptr{Float64}), "./", filename="jsclinsolve$N")

The first two compile_shlibs work as intended, the last produces garbage.

The namespacing syntax we get from getproperty(::Type, ::Symbol) is nifty, but the price seems too high. Could we get the option, perhaps as a kwarg for @adt, to not emit this method?

[Roger-luo]

Ah, thanks, Chris! James warned me about this during JuliaCon2023, actually, lol, but I've written this package with this, so it's a bit hard to change that in this package. But the good news is, because this also hurts me, I'm actually about to release a new package to replace the ADT implemented in this package, which does not overload the getproperty but mimic the namespace with an actual module.

I just made the repo public: https://github.com/Roger-luo/Moshi.jl

(I haven't written many tests yet), here is a similar example in Expronicon

julia> using Moshi.Data.Prelude

julia> @data Foo begin
           Bar
           Baz(Int)
       end
Main.Foo

julia> Foo.Ba
Bar
Baz
julia> Foo.Ba
Bar
Baz
julia> Foo.Baz(2)
Main.Foo.Type(Main.Foo.var"#Foo#Storage"((0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00), (0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00), ()))

The main difference is the type of Foo is not Foo itself anymore, it's Foo.Type. So Foo is nothing fancy but just a module. This should make Julia's compiler happier. And no need for the public thing, one can just use Julia's own namespace semantics.

[chriselrod]

I've written this package with this, so it's a bit hard to change that in this package.

I've currently pinned Expronicon to 0.8, which doesn't have this problem https://github.com/Roger-luo/Expronicon.jl/blob/v0.8.5/src/adt/emit.jl For now, that's an easy fix.

Moshi looks interesting, though, so I'll look into that. Thanks!

[Roger-luo]

Yeah, that was because the ADT was not implemented in a memory-efficient way back then. I think a few things changed afterwards which was breaking. I learned this from Mason in SumTypes (which now uses Unions instead). I'm planning to provide both in Moshi, one for C-like tagged Union, and the other memory layout for Julia native optimization.

[Roger-luo]

closing in favor of https://github.com/Roger-luo/Moshi.jl/