Turing.jl icon indicating copy to clipboard operation
Turing.jl copied to clipboard

DynamicPPL -> 0.29; Julia -> 1.10; Tapir -> Mooncake

Open penelopeysm opened this issue 1 year ago β€’ 12 comments

This PR bumps DynamicPPL to 0.29.

Julia

This necessitates dropping support for Julia < 1.10

  • the last version of SciMLBase to support Julia 1.9 is 2.24.0
    • which in turn requires ADTypes 0.2
  • but DynamicPPL 0.28.3 (and by extension 0.29) requires ADTypes 1.0 (https://github.com/TuringLang/DynamicPPL.jl/pull/638)

Mooncake

Likewise, I've replaced Tapir -> Mooncake as only the latter has true support for DynamicPPL 0.29.

penelopeysm avatar Sep 25 '24 11:09 penelopeysm

Codecov Report

Attention: Patch coverage is 72.72727% with 6 lines in your changes missing coverage. Please review.

Project coverage is 86.39%. Comparing base (24e6870) to head (9fb9e95). Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/mcmc/mh.jl 70.00% 6 Missing :warning:
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #2341       +/-   ##
===========================================
+ Coverage    0.00%   86.39%   +86.39%     
===========================================
  Files          22       22               
  Lines        1533     1573       +40     
===========================================
+ Hits            0     1359     +1359     
+ Misses       1533      214     -1319     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Sep 25 '24 12:09 codecov[bot]

Pull Request Test Coverage Report for Build 11466651176

Details

  • 16 of 22 (72.73%) changed or added relevant lines in 3 files are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage increased (+86.4%) to 86.395%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/mcmc/mh.jl 14 20 70.0%
<!-- Total: 16 22
Files with Coverage Reduction New Missed Lines %
src/stdlib/RandomMeasures.jl 1 33.33%
<!-- Total: 1
Totals Coverage Status
Change from base Build 11314008798: 86.4%
Covered Lines: 1359
Relevant Lines: 1573

πŸ’› - Coveralls

coveralls avatar Sep 25 '24 14:09 coveralls

Tests still aren't passing, there are still a few changes that need to be made to bring it up to date with https://github.com/TuringLang/DynamicPPL.jl/pull/575.

I don't mind handling this, but it will take me a long time, so if anybody else feels like expediting it please feel free to jump in:)

penelopeysm avatar Sep 26 '24 23:09 penelopeysm

I'll ask around if anyone objects to dropping <1.9.

mhauru avatar Sep 30 '24 14:09 mhauru

In the meantime I can work to get the PR ready to merge, we will need it at some point anyway whether it's sooner or later:)

penelopeysm avatar Sep 30 '24 14:09 penelopeysm

As it turns out, Julia 1.9 won't work either, because SciMLBase.

(see e.g. https://github.com/TuringLang/Turing.jl/actions/runs/11153561181/job/31001300867)

penelopeysm avatar Oct 03 '24 00:10 penelopeysm

I think everything almost works, except for a Mooncake-or-further-upstream bug which I'll report tomorrow.

Edit: https://github.com/compintell/Mooncake.jl/issues/283

penelopeysm avatar Oct 03 '24 23:10 penelopeysm

@yebai / @mhauru Since we're making fairly sweeping changes here, is it an opportune time to remove Tracker support as well? https://github.com/TuringLang/Turing.jl/issues/2356

penelopeysm avatar Oct 04 '24 09:10 penelopeysm

@torfjelde Could I get you to take a look at my changes to src/mcmc please? The tests all pass, but I'm a little anxious πŸ˜„

Also, this PR fixes the following bug, but not entirely sure what's the best way to write a test for it. (Other samplers already work correctly with this, it's just MH that fails.) Would it make sense to extend test/test_utils/models.jl and test/test_utils/numerical_tests.jl?

julia> using Turing

julia> @model f() = x ~ LKJCholesky(2, 1)
f (generic function with 2 methods)

julia> sample(f(), MH(), 100)
ERROR: MethodError: no method matching length(::LinearAlgebra.Cholesky{Float64, Matrix{Float64}})
[...]

penelopeysm avatar Oct 04 '24 09:10 penelopeysm

(Other samplers already work correctly with this, it's just MH that fails.) Would it make sense to extend test/test_utils/models.jl and test/test_utils/numerical_tests.jl?

I'd just add it to the MH specific tests for now :+1:

torfjelde avatar Oct 04 '24 12:10 torfjelde

@yebai / @mhauru Since we're making fairly sweeping changes here, is it an opportune time to remove Tracker support as well? https://github.com/TuringLang/Turing.jl/issues/2356

It sounds good to me.

yebai avatar Oct 08 '24 17:10 yebai

This upgrade is also blocked by https://github.com/TuringLang/docs/issues/521 – ideally we want to make sure any relevant docs build with Mooncake. Because we don't want to upgrade to 0.35.0, patch Mooncake to work for the docs, and then release 0.35.1 with a new compat entry for Mooncake just so that the docs work.

However, the Bayesian NN example errors with Mooncake. Will create specific upstrea issues etc when I get the time

penelopeysm avatar Oct 08 '24 22:10 penelopeysm

Once tests pass (hopefully soon πŸ™), still TODO:

  • [x] Update API docs
  • [x] Changelog note

penelopeysm avatar Oct 22 '24 14:10 penelopeysm

CI is finally passing, so it's time to ask for you all to take a final look!

To be honest, there are no major code changes since the last round of reviews. Most of the CI errors were related to upstream issues which have since been fixed (mostly due to @willtebbutt's heroic efforts πŸ™Œ ). Julia 1.11 coming out while this PR was open definitely also made it more fun πŸ˜„

There are a few outstanding TODOs which I've opened issues for:

  • #2369
  • #2371
  • #2372

However, this PR makes some sweeping changes to the library. To protect this PR from scope creep, I think we should address these in separate PRs. Notably, all of these issues are about tests, not the library code itself.

penelopeysm avatar Oct 22 '24 23:10 penelopeysm

Great work. Many thanks, @penelopeysm and all!

yebai avatar Oct 23 '24 16:10 yebai

great work! thanks guys

sunxd3 avatar Oct 23 '24 19:10 sunxd3