ChainRules.jl icon indicating copy to clipboard operation
ChainRules.jl copied to clipboard

Glue packages with working Pkg compat versioning

Open oxinabox opened this issue 5 years ago • 3 comments

This is the outcome of a discussion I had with @KristofferC about how to make glue packages work.

We should put it into practice for SpecialFunctions.jl and NaNMath.jl

We make a new package called: ChainRules_SpecialFunctions_Glue.jl which has the released with compat on SpecialFunctions.jl declared then in ChainRules.jl we would declare a dependency NOT on SpecialFunctions but on ChainRules_SpecialFunctions_Glue.jl and then in the source code of ChainRules.jl would do:

    @require SpecialFunctions="276daf66-3868-5448-9aa4-cd146d93841b" begin
        using ChainRules_SpecialFunctions_Glue  # maybe use `Base.require` instead
    end

So ChainRules.jl never has a direct dependency on SpecialFunctions.jl only on ChainRules_SpecialFunctions_Glue.jl. But ChainRules.jl does use Require to conditionally load ChainRules_SpecialFunctions_Glue.jl in response to SpecialFunction.jl being loaded.

and ChainRules_SpecialFunctions_Glue.jl does have a direct dependency on SpecialFunctions.jl (and on ChainRulesCore.jl), and it declares constraints on it so the pakcage manager can workout which should be loaded.

In pacticular, ChainRules_SpecialFunctions_Glue.jl would not support both SpecialFunctions.jl v0.7 and v0.8 at the same time. Rather ChainRules_SpecialFunctions_Glue.jl v1.0.0 would support SpecialFunctions.jl v0.7 and ChainRules_SpecialFunctions_Glue.jl v1.1.0 would support SpecialFunctions.jl v0.8

And ChainRules.jl would support ChainRules_SpecialFunctions_Glue.jl 1.x.y

TL; DR: you use Requires on the Other package, to load the Glue and you put the Glue package in your dependencies

oxinabox avatar Oct 12 '20 15:10 oxinabox

Is it decided that this won't just go in SpecialFunctions.jl itself (https://github.com/JuliaMath/SpecialFunctions.jl/pull/238)?

nickrobinson251 avatar Oct 12 '20 15:10 nickrobinson251

A more wild idea for making it an optional dependency. What we want is if a new release of SpecialFunctions.jl comes out and people end up restricted to it, then rather than the package manager erroring, we want a warning message displayed, and for it to just not load any of the glue code.

I think we can accomplish this via making a release of ChainRules_SpecialFunctions_Glue.jl

  • with a really low version number like v0.0.1, so that the package manager never resolved to it unless it has no other option,
  • with no restrictions on SpecialFunctions.jl , i.e in compat put >=0 (or maybe to trick auto-merge <=10000?)
  • with the code content just __init__() = @warn "No compatible versions ChainRules_SpecialFunctions_Glue
  • ChainRules.jl would then declare that it does want it to be an Optional depenency by puting 0.0.1 into the compat for ChainRules_SpecialFunctions_Glue.jl

does this work? cc @StefanKarpinski

oxinabox avatar Oct 12 '20 16:10 oxinabox

I am going to keep this open. while we now have moved out SpecialFunctions and have just plain deleted NaNMath, the root desire still remains. E.g. this is going to come up for Distences.jl probably.

oxinabox avatar Jun 02 '21 12:06 oxinabox