StructuralIdentifiability.jl
StructuralIdentifiability.jl copied to clipboard
ModelingToolkit extension completely freezing Julia
using ModelingToolkit
using StructuralIdentifiability
const SI = StructuralIdentifiability
function testSI()
t = ModelingToolkit.t_nounits
D = ModelingToolkit.D_nounits
@variables X(t)
@parameters p d
eqs = [
D(X) ~ p - d*X
]
@mtkbuild osys = ODESystem(eqs, t)
measured_quantities = [X]
funcs_to_check = [osys.p]
SI.assess_identifiability(osys; measured_quantities, funcs_to_check)
end
causes Julia to completely hang (as reported on Slack by @TorkelE).
Note that changing to funcs_to_check = [p] fixes the problem.
Environment:
(StructuralIdentifiability) pkg> st
Project StructuralIdentifiability v0.5.9
Status `~/.julia/dev/StructuralIdentifiability/Project.toml`
⌅ [c3fe647b] AbstractAlgebra v0.41.11
[861a8166] Combinatorics v1.0.2
[864edb3b] DataStructures v0.18.20
⌅ [0b43b601] Groebner v0.7.5
[c8e1da08] IterTools v1.10.0
[1914dd2f] MacroTools v0.5.13
⌅ [2edaba10] Nemo v0.45.7
⌅ [3e851597] ParamPunPam v0.4.1
[aea7be01] PrecompileTools v1.2.1
[27ebfcd6] Primes v0.5.6
[a759f4b9] TimerOutputs v0.5.24
[ade2ca70] Dates
[37e2e46d] LinearAlgebra
[56ddb016] Logging
[9a3f8284] Random
Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated`
on Julia 1.10.5.
Many thanks for reporting ! I am looking into this
I just tried with Julia 1.10.0 and was not able to reproduce the problem: both versions (osys.p and p) work fine. I will try to upgrade my Julia to see if I can get the hanging behaviour.
Environment:
Status `~/.julia/environments/v1.10/Project.toml`
⌅ [c3fe647b] AbstractAlgebra v0.41.11
⌅ [66b61cbe] AlgebraicSolving v0.4.16
[4c88cf16] Aqua v0.8.7
[6e4b80f9] BenchmarkTools v1.5.0
[0c46a032] DifferentialEquations v7.14.0
[7c1d4256] DynamicPolynomials v0.6.0
[60bf3e95] GLPK v1.2.1
⌅ [0b43b601] Groebner v0.7.5
[7073ff75] IJulia v1.25.0
[c8e1da08] IterTools v1.10.0
[682c06a0] JSON v0.21.4
[98e50ef6] JuliaFormatter v1.0.60
[b964fa9f] LaTeXStrings v1.3.1
[16fef848] LiveServer v1.3.1
⌃ [291d046c] MixedSubdivisions v1.1.4
⌃ [961ee093] ModelingToolkit v9.39.1
⌅ [2edaba10] Nemo v0.45.7
[bac558e1] OrderedCollections v1.6.3
[f1435218] Oscar v1.1.1
⌅ [3e851597] ParamPunPam v0.4.1
[91a5bcdd] Plots v1.40.8
[67491407] Polyhedra v0.7.8
[731186ca] RecursiveArrayTools v3.27.0
⌃ [295af30f] Revise v3.5.18
[276daf66] SpecialFunctions v2.4.0
[2913bbd2] StatsBase v0.34.3
[220ca800] StructuralIdentifiability v0.5.9 `~/.julia/dev/StructuralIdentifiability`
⌃ [d1185830] SymbolicUtils v3.6.0
[98d24dd4] TestSetExtensions v3.0.0
[9a3f8284] Random
[8dfed614] Test
Two observations so far:
- there is no problem in Julia
1.10.0but there is on1.10.5 - if the
osys.pversion, if I interrupt by Ctrl+C, then the structural identifiability computation runs smoothly
It seems like it is hanging due to some kind of type inference. Cutting the compiler shows that the compiler is hanging in specialize_method, which does some type inference. When I snoop the compiling process it seems like it is compiling reduce_ode_mod_p when it hangs (I called this with a ReactionSystem but essentially the same trace shows up if you call it with an ODESystem).
17-element Vector{Core.Compiler.Timings.Timing}:
Core.Compiler.Timings.Timing(InferenceFrameInfo for Core.Compiler.Timings.ROOT()) with 0 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for StructuralIdentifiability.assess_identifiability(::ReactionSystem{Catalyst.NetworkProperties{Int64, SymbolicUtils.BasicSymbolic{Real}}})) with 1 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for CatalystStructuralIdentifiabilityExtension.var"#assess_identifiability#3"(::Vector{Any}, ::Vector{Any}, ::Vector{Any}, ::Bool, ::Bool, ::Base.Pairs{Symbol, Union{}, Tuple{}, @NamedTuple{}}, ::typeof(assess_identifiability), ::ReactionSystem{Catalyst.NetworkProperties{Int64, SymbolicUtils.BasicSymbolic{Real}}})) with 7 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for Core.kwcall(::NamedTuple{(:measured_quantities, :funcs_to_check), <:Tuple{Vector{Equation}, Any}}, ::typeof(assess_identifiability), ::ReactionSystem)) with 1 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for CatalystStructuralIdentifiabilityExtension.var"#assess_identifiability#3"(::Vector{Equation}, ::Vector{Any}, ::Any, ::Bool, ::Bool, ::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}}, ::typeof(assess_identifiability), ::ReactionSystem)) with 7 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for Core.kwcall(::NamedTuple, ::typeof(assess_identifiability), ::ReactionSystem)) with 0 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for CatalystStructuralIdentifiabilityExtension.var"#assess_identifiability#3"(::Any, ::Any, ::Any, ::Any, ::Any, ::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}}, ::typeof(assess_identifiability), ::ReactionSystem)) with 7 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for Core.kwcall(::NamedTuple, assess_identifiability::typeof(assess_identifiability), ::ODE{P}) where P<:AbstractAlgebra.MPolyRingElem{Nemo.QQFieldElem}) with 0 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for StructuralIdentifiability.var"#assess_identifiability#694"(::Any, ::Vector{<:Union{AbstractAlgebra.Generic.FracFieldElem{P}, P}}, ::Float64, ::Any, assess_identifiability::typeof(assess_identifiability), ::ODE{P}) where P<:AbstractAlgebra.MPolyRingElem{Nemo.QQFieldElem}) with 1 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for Base.CoreLogging.with_logger(::StructuralIdentifiability.var"#695#697"{_A, Vector{var"#s1850"}, Float64, ODE{P}} where {_A, P<:AbstractAlgebra.MPolyRingElem{Nemo.QQFieldElem}, var"#s1850"<:Union{AbstractAlgebra.Generic.FracFieldElem{P}, P}, P<:AbstractAlgebra.MPolyRingElem{Nemo.QQFieldElem}}, ::Logging.ConsoleLogger)) with 0 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for Base.CoreLogging.with_logstate(::StructuralIdentifiability.var"#695#697"{_A, Vector{var"#s1850"}, Float64, ODE{P}} where {_A, P<:AbstractAlgebra.MPolyRingElem{Nemo.QQFieldElem}, var"#s1850"<:Union{AbstractAlgebra.Generic.FracFieldElem{P}, P}, P<:AbstractAlgebra.MPolyRingElem{Nemo.QQFieldElem}}, ::Base.CoreLogging.LogState)) with 2 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for (::StructuralIdentifiability.var"#695#697"{_A, Vector{var"#s1850"}, Float64, ODE{P}} where {_A, P<:AbstractAlgebra.MPolyRingElem{Nemo.QQFieldElem}, var"#s1850"<:Union{AbstractAlgebra.Generic.FracFieldElem{P}, P}, P<:AbstractAlgebra.MPolyRingElem{Nemo.QQFieldElem}})()) with 1 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for Core.kwcall(::NamedTuple{(:funcs_to_check, :prob_threshold), <:Tuple{Any, Float64}}, _assess_identifiability::typeof(StructuralIdentifiability._assess_identifiability), ::ODE{P} where P<:AbstractAlgebra.MPolyRingElem{Nemo.QQFieldElem})) with 0 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for StructuralIdentifiability.var"#_assess_identifiability#699"(::Any, ::Float64, _assess_identifiability::typeof(StructuralIdentifiability._assess_identifiability), ::ODE{P} where P<:AbstractAlgebra.MPolyRingElem{Nemo.QQFieldElem})) with 9 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for Core.kwcall(::NamedTuple{(:funcs_to_check, :prob_threshold, :type, :trbasis), <:Tuple{Any, Float64, Symbol, Vector{Nemo.QQMPolyRingElem}}}, _assess_local_identifiability::typeof(StructuralIdentifiability._assess_local_identifiability), ::ODE{P} where P<:AbstractAlgebra.MPolyRingElem{Nemo.QQFieldElem})) with 1 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for StructuralIdentifiability.var"#_assess_local_identifiability#198"(::Vector, ::Float64, ::Symbol, ::Vector{Nemo.QQMPolyRingElem}, ::Vector{Any}, _assess_local_identifiability::typeof(StructuralIdentifiability._assess_local_identifiability), ::ODE{P}) where P<:AbstractAlgebra.MPolyRingElem{Nemo.QQFieldElem}) with 42 children
Core.Compiler.Timings.Timing(InferenceFrameInfo for StructuralIdentifiability.reduce_ode_mod_p(::ODE{<:AbstractAlgebra.MPolyRingElem{Nemo.QQFieldElem}}, ::Int64)) with 6 children
The timings before I cut it are here:
[Int(timing.time) for timing in timings]
17-element Vector{Int64}:
113583
305501
659876
248916
278582
243250
513582
759375
397000
220875
341249
357750
628750
1068958
1226791
2985504
680757720458
Many thanks ! I will look closer into this function
Could be related : https://github.com/SciML/StructuralIdentifiability.jl/issues/362
We are now seeing it crashing / hanging CI when just loading the extension on 1.11:
https://github.com/SciML/Catalyst.jl/actions/runs/11239742228/job/31248152175?pr=1053#step:6:1197
CI crash on 1.11 seems to be a bug in Julia from https://github.com/JuliaLang/julia/issues/56040.
The freeze from the current issue seems to be of different nature: it manifests in 1.10.5 and the offending commit from https://github.com/JuliaLang/julia/issues/56040 is not present in 1.10.5.
@pogudingleb How do we debug the freeze in the current issue? The functions reduce_ode_mod_p and friends hinted by @vyudu seem as legal Julia code to me.
Very elusive behaviour. Even if I take out the original code from the function:
using ModelingToolkit
using StructuralIdentifiability
const SI = StructuralIdentifiability
t = ModelingToolkit.t_nounits
D = ModelingToolkit.D_nounits
@variables X(t)
@parameters p d
eqs = [
D(X) ~ p - d*X
]
@mtkbuild osys = ODESystem(eqs, t)
measured_quantities = [X]
funcs_to_check = [osys.p]
SI.assess_identifiability(osys; measured_quantities, funcs_to_check)
It also works fine ! But being enclosed in a function, it hangs...
Hi, is there any update on this? Is it fixed on 1.10.6 and 1.11.2 (I know the latter isn't released yet, but given it should be soon that would close this issue I think)?
(I'm just trying to figure out if we can reenable the Catalyst SI extension for our next release or will have to keep it disabled.)
Sorry, I was busy recently and the bug is tricky so the several hours I could spend were not enough to understand the problem. I will double check that it works on 1.10.6 and 1.11.2 and, if yes, close the issue. It seems that the SI code is completely legal and was not precompiling correctly for some extraterrestrial reason.
Nope, it hangs as well on 1.10.6.
Ah, darn. Thanks for checking!
FWIW, I just checked and we are still seeing this issue on 1.11.2 also, so it isn't just a 1.10 issue.
@isaacsas Thanks for checking!
I have just tried on the nightly build which is version 1.12.0-DEV.1867, and everything works there. I hope these changes were back ported, I will try the current master of 1.10 and 1.11
That is really great to hear, and yes, hopefully it is backported! Even if it just gets into 11.3 we could add the SI extension back into Catalyst. We just need the ability to run our CI tests on the latest Julia version so we know there aren't any bugs on our end.
I have just checked the pre-release version of 1.10.8 and 1.11.3. On 1.11.3 everything works but still freezes in 1.10.8. I have created an issue
Awesome to hear! Would be great to have the extension back and working for when we release Catalyst v15
Confirming: everything works on the (just released) 1.11.3
Amazing to hear this is completed.
The Catalyst tests are still stalling, but given that this works now hopefully it is something I can figure out soon!
The Catalyst tests are still stalling, but given that this works now hopefully it is something I can figure out soon!
Are you testing on 1.10.8 or 1.11.3? In the former, the problem persists.
Ok, the test here works on v1.11.3. However, loading Catalyst causes it to fail. Very weird. Example:
This works:
julia> VERSION
v"1.11.3"
(Environment - Temporary) pkg> st
Status `~/Desktop/Julia Playground/Environment - Temporary/Project.toml`
[479239e8] Catalyst v14.4.1 `~/.julia/dev/Catalyst`
⌃ [961ee093] ModelingToolkit v9.59.0
[220ca800] StructuralIdentifiability v0.5.12
[56ddb016] Logging v1.11.0
Info Packages marked with ⌃ have new versions available and may be upgradable.
julia> using ModelingToolkit
julia> using StructuralIdentifiability
julia> const SI = StructuralIdentifiability
StructuralIdentifiability
julia> function testSI_MTK()
t = ModelingToolkit.t_nounits
D = ModelingToolkit.D_nounits
@variables X(t)
@parameters p d
eqs = [
D(X) ~ p - d*X
]
@mtkbuild osys = ODESystem(eqs, t)
measured_quantities = [X]
funcs_to_check = [osys.p]
SI.assess_identifiability(osys; measured_quantities, funcs_to_check)
end
testSI_MTK (generic function with 1 method)
julia> testSI_MTK()
┌ Info: System parsed into X' = -X*d + p
└ y1 = X
[ Info: Assessing local identifiability
[ Info: Assessing global identifiability
[ Info: Computing IO-equations
[ Info: Computed IO-equations in 4.007655171 seconds
[ Info: Computing Wronskians
[ Info: Computed Wronskians in 1.668636723 seconds
[ Info: Dimensions of the Wronskians [3]
[ Info: Ranks of the Wronskians computed in 0.018164206 seconds
[ Info: Global identifiability assessed in 10.423182398 seconds
OrderedCollections.OrderedDict{Num, Symbol} with 1 entry:
p => :globally
julia> testSI_MTK()
┌ Info: System parsed into X' = -X*d + p
└ y1 = X
[ Info: Assessing local identifiability
[ Info: Assessing global identifiability
[ Info: Computing IO-equations
[ Info: Computed IO-equations in 0.000720843 seconds
[ Info: Computing Wronskians
[ Info: Computed Wronskians in 0.000506108 seconds
[ Info: Dimensions of the Wronskians [3]
[ Info: Ranks of the Wronskians computed in 8.602e-6 seconds
[ Info: Global identifiability assessed in 0.007127755 seconds
OrderedCollections.OrderedDict{Num, Symbol} with 1 entry:
p => :globally
However, inserting a using Catalyst before the second testSI_MTK() call messes everything up:
julia> VERSION
v"1.11.3"
(Environment - Temporary) pkg> st
Status `~/Desktop/Julia Playground/Environment - Temporary/Project.toml`
[479239e8] Catalyst v14.4.1 `~/.julia/dev/Catalyst`
⌃ [961ee093] ModelingToolkit v9.59.0
[220ca800] StructuralIdentifiability v0.5.12
[56ddb016] Logging v1.11.0
Info Packages marked with ⌃ have new versions available and may be upgradable.
julia> using ModelingToolkit
julia> using StructuralIdentifiability
julia> const SI = StructuralIdentifiability
StructuralIdentifiability
julia> function testSI_MTK()
t = ModelingToolkit.t_nounits
D = ModelingToolkit.D_nounits
@variables X(t)
@parameters p d
eqs = [
D(X) ~ p - d*X
]
@mtkbuild osys = ODESystem(eqs, t)
measured_quantities = [X]
funcs_to_check = [osys.p]
SI.assess_identifiability(osys; measured_quantities, funcs_to_check)
end
testSI_MTK (generic function with 1 method)
julia> testSI_MTK()
┌ Info: System parsed into X' = -X*d + p
└ y1 = X
[ Info: Assessing local identifiability
[ Info: Assessing global identifiability
[ Info: Computing IO-equations
[ Info: Computed IO-equations in 5.023534383 seconds
[ Info: Computing Wronskians
[ Info: Computed Wronskians in 1.786881821 seconds
[ Info: Dimensions of the Wronskians [3]
[ Info: Ranks of the Wronskians computed in 0.021605133 seconds
[ Info: Global identifiability assessed in 12.305836099 seconds
OrderedCollections.OrderedDict{Num, Symbol} with 1 entry:
p => :globally
julia> using Catalyst
julia> testSI_MTK()
Here it just stalls.
Finally, if I comment out the Catalyst Structural Identifiability extension it starts working fine again.
Looking further into it. Just removing the CatalystStructuralIdentifiabilityExtension's overload to SI.assess_identifiability solves everything. This function does not actually gets called though, however, somehow its present causes everything to stall.
Some further comment, just on how odd everything is. This is the current CatalystStructuralIdentifiabilityExtension's assess_identifiability overload:
function SI.assess_identifiability(rs::ReactionSystem, args...;
measured_quantities = [], known_p = [], funcs_to_check = Vector(),
remove_conserved = true, ignore_no_measured_warn = false, kwargs...)
# Creates a ODESystem, list of measured quantities, and functions to check, of SI's preferred form.
osys, conseqs, consconsts, vars = make_osys(rs; remove_conserved)
measured_quantities = make_measured_quantities(rs, measured_quantities, known_p,
conseqs; ignore_no_measured_warn)
funcs_to_check = make_ftc(funcs_to_check, conseqs, vars)
# Computes identifiability and converts it to a easy to read form.
out = SI.assess_identifiability(osys, args...; measured_quantities,
funcs_to_check, kwargs...)
return make_output(out, funcs_to_check, consconsts)
end
using It, everything stalls.
Removing the last bit where we call SI.assess_identifiability and everything works fine:
function SI.assess_identifiability(rs::ReactionSystem, args...;
measured_quantities = [], known_p = [], funcs_to_check = Vector(),
remove_conserved = true, ignore_no_measured_warn = false, kwargs...)
# Creates a ODESystem, list of measured quantities, and functions to check, of SI's preferred form.
osys, conseqs, consconsts, vars = make_osys(rs; remove_conserved)
measured_quantities = make_measured_quantities(rs, measured_quantities, known_p,
conseqs; ignore_no_measured_warn)
funcs_to_check = make_ftc(funcs_to_check, conseqs, vars)
# Computes identifiability and converts it to a easy to read form.
#out = SI.assess_identifiability(osys, args...; measured_quantities,
# funcs_to_check, kwargs...)
return 1
end
Actually, adding these errors messages prevents the stalling (but no errors are thrown):
function SI.assess_identifiability(rs::ReactionSystem, args...;
measured_quantities = [], known_p = [], funcs_to_check = Vector(),
remove_conserved = true, ignore_no_measured_warn = false, kwargs...)
# Creates a ODESystem, list of measured quantities, and functions to check, of SI's preferred form.
osys, conseqs, consconsts, vars = make_osys(rs; remove_conserved)
measured_quantities = make_measured_quantities(rs, measured_quantities, known_p,
conseqs; ignore_no_measured_warn)
funcs_to_check = make_ftc(funcs_to_check, conseqs, vars)
# Computes identifiability and converts it to a easy to read form.
error("We should not be here")
out = SI.assess_identifiability(osys, args...; measured_quantities,
funcs_to_check, kwargs...)
error("Nor should we be should not be here")
return make_output(out, funcs_to_check, consconsts)
end
Sorry for the mess, will continue looking through this, and get back here if I have something useful.
What still seems to be happening is that the code seems to run until the end, but refuses to actually return anything. Running the above workflow, but adding some code into the SI assess_identifiability function (ModelingToolkitSIExt.jl line 415):
println("Here 1")
out = OrderedDict(nemo2mtk[param] => result[param] for param in funcs_to_check_)
println("Here 2")
return out
println("Here 3")
Here, we reach the second print statement and then returns the output. In the second case there is not any printed output until I forde an exit through ctrl+c, at which point int time we actually get everything printed out as normal:
julia> using ModelingToolkit
julia> using StructuralIdentifiability
Precompiling ModelingToolkitSIExt...
1 dependency successfully precompiled in 30 seconds. 319 already precompiled.
julia> const SI = StructuralIdentifiability
StructuralIdentifiability
julia> function testSI_MTK()
t = ModelingToolkit.t_nounits
D = ModelingToolkit.D_nounits
@variables X(t)
@parameters p d
eqs = [
D(X) ~ p - d*X
]
@mtkbuild osys = ODESystem(eqs, t)
measured_quantities = [X]
funcs_to_check = [osys.p]
SI.assess_identifiability(osys; measured_quantities, funcs_to_check)
end
testSI_MTK (generic function with 1 method)
julia> testSI_MTK()
┌ Info: System parsed into X' = -X*d + p
└ y1 = X
[ Info: Assessing local identifiability
[ Info: Assessing global identifiability
[ Info: Computing IO-equations
[ Info: Computed IO-equations in 4.795254909 seconds
[ Info: Computing Wronskians
[ Info: Computed Wronskians in 1.724396875 seconds
[ Info: Dimensions of the Wronskians [3]
[ Info: Ranks of the Wronskians computed in 0.016648773 seconds
[ Info: Global identifiability assessed in 11.739331163 seconds
Here 1
Here 2
OrderedCollections.OrderedDict{Num, Symbol} with 1 entry:
p => :globally
julia> using Catalyst
julia> testSI_MTK()
^C^C^C^C^C^C^C^CWARNING: Force throwing a SIGINT
Internal error: during type inference of
testSI_MTK()
Encountered unexpected error in runtime:
InterruptException()
segv_handler at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/signals-unix.c:356
┌ Info: System parsed into X' = -X*d + p
└ y1 = X
[ Info: Assessing local identifiability
[ Info: Assessing global identifiability
[ Info: Computing IO-equations
[ Info: Computed IO-equations in 0.002162717 seconds
[ Info: Computing Wronskians
[ Info: Computed Wronskians in 0.001311264 seconds
[ Info: Dimensions of the Wronskians [3]
[ Info: Ranks of the Wronskians computed in 1.9153e-5 seconds
[ Info: Global identifiability assessed in 0.00601437 seconds
Here 1
Here 2
OrderedCollections.OrderedDict{Num, Symbol} with 1 entry:
p => :globally
julia>
Another weird thing, this only seems to be a problem for assess_identifiability. For the other functions it works fine.
Hi @TorkelE. Thank you very much for your effort in hunting these strange bugs. A question: do you know which commit in JuliaLang/julia makes this bug appear?
Ok, this is just strange. There is a discussion here: https://discourse.julialang.org/t/how-to-debug-an-error-in-type-inference/126122/8 which mostly shows what might be going on. However, to actually figure it out from here basically requires a deep dive into compiler stuff. In practise I have found another solution though (basically adding type assortments, which seems to help type inference enough that we do not encounter this issue). So if nothing unforeseen happens the SI extension should be back up and running at the next Catalyst release.