JuliaDiff
Replace this package with ForwardDiff's `Dual`?
It's annoying to have to port functionality between these two packages. It would be better if there were only be one Dual implementation that development efforts can focus on, if we can get away with it. AFAIK, ForwardDiff's Dual number is strictly superior to the version implemented in this package - in addition to allowing nested dual (i.e. hyper-dual) numbers and N partials derivatives at once, it features the same performance as this package's Dual in the single-element case, and is just as easy to write new derivative definitions for.
I propose we pull out ForwardDiff's Dual type and place it here instead. Of course, we'd want to ensure that we include any derivative definitions here that are missing from ForwardDiff, and vice versa.
The only controversial part of this change is that ForwardDiff's Dual is a subtype of Real, whereas the Dual here is a subtype of Number. I would require that we keep the proposed Dual a subtype of Real, since the goal of a dual number implementation is to programmatically support the Real interface. I really don't think this will matter in any practical sense, but others may disagree.
I would require that we keep the proposed Dual a subtype of Real, since the goal of a dual number implementation is to programmatically support the Real interface.
Dual already supports complex numbers (https://github.com/JuliaDiff/DualNumbers.jl/pull/29) so not sure if that would keep everyone happy.
I would also argue for the educational benefit of keeping such a simple implementation around. The extra layer of indirection with Partials makes ForwardDiff's Dual type much less suitable for explaining in a class.
Dual already supports complex numbers (#29) so not sure if that would keep everyone happy.
I don't think this is an issue, ForwardDiff's Dual also supports complex numbers (and by extension, multidimensional dual-complex numbers). The only difference between the two implementations is the arbitrary choice of orientation; Complex{T<:Dual} (ForwardDiff) rather than Dual{T<:Complex} (current behavior here). For example:
Dual{T}(c::Complex{T}) = Complex(Dual(real(c), one(T)), Dual(imag(c), one(T)))
I would also argue for the educational benefit of keeping such a simple implementation around. The extra layer of indirection with Partials makes ForwardDiff's Dual type much less suitable for explaining in a class.
I've never had trouble explaining it to people, but even so, you can easily define a simple dual number (both mathematically and your own implementation) in two slides - why would you need a whole package that duplicates code and divides development efforts just to teach dual numbers to people? If you need them to use the package, just say "This package uses an implementation of Dual that supports more simultaneous partial derivatives, but is equivalent for the single derivative case." The development benefit of this decision far outweighs the educational cost (if there even is any).
And just keep in mind, since complexity is a discussion topic: it's just as easy to write new derivative definitions for ForwardDiff.Dual as it is for DualNumbers.Dual. You're still just multiplying the derivative w.r.t. the value by the derivative component. The only difference is that the name of the derivative component field is different.
Are there any downsides of Complex{T<:Dual} versus Dual{T<:Complex}? A quick check shows that exp and real behave equivalently to DualNumbers implementation.
The show override is not as nice, though.
@MikaelSlevinsky might also be interested.
Are there any downsides of Complex{T<:Dual} versus Dual{T<:Complex}?
Of course, I'm biased towards Complex{T<:Dual}, but I believe they're equivalent interpretations. AFAICT, the big downside of Dual{T<:Complex} is that your dual number implementation has to support both the Complex and Real interfaces, whereas Complex{T<:Dual} implies that your dual number only needs to support the Real interface.
As previously mentioned, another obvious upside for ForwardDiff.Dual vs. DualNumbers.Dual is that all its extra features should work for complex differentiation as well.
I think the reason it uses Dual{T<:Complex} was that it was felt that Dual <: Real was 'wrong' in some sense, and without that, you can't do Complex{T<:Dual}.
But perhaps the approach of ForwardDiff.Dual is better: maybe <: Real just means override the right methods, not that it is subset of real numbers in the mathematical sense.
+1 for this suggestion. The argument about that this package is easy to explain to a class doesn't hold much water to me. A small DualNumbersEducation.jl package could be made in half an hour. For the "official" DualNumbers.jl package we should in my opinion have Dual numbers that are best from a usage point of view.
Is the proposal that ForwardDiff.Dual be renamed DualNumbers.Dual, that is, we retain this package but with a new implementation?
Is the proposal that ForwardDiff.Dual be renamed DualNumbers.Dual, that is, we retain this package but with a new implementation?
Yes (sorry for the late reply).
Sorry I'm late to the conversation...
I'm guessing one pro for FowardDiff.Dual <: Real is that it sneaks by Base's restriction of Complex{T<:Real} and I'm assuming a lot is gained for free by doing this?
Due to that same Base restriction, making ForwardDiff.Dual <: Number implies supporting Dual{T<:Complex} rather than Complex{T<:Dual}, since the latter would be impossible. What are the cons? Are there breaking changes?
Risking sounding like a broken record, dual numbers extend their underlying field. From an abstract view, it seems unreasonable to subtype them as reals.
I'm guessing one pro for FowardDiff.Dual <: Real is that it sneaks by Base's restriction of Complex{T<:Real} and I'm assuming a lot is gained for free by doing this? Due to that same Base restriction, making ForwardDiff.Dual <: Number implies supporting Dual{T<:Complex} rather than Complex{T<:Dual}, since the latter would be impossible.
Yup - this is what I meant by "forcing the better orientation" in earlier discussions.
What are the cons? Are there breaking changes?
AFAICT, the only con is that - besides the fact that it doesn't need to support the whole Number interface - ForwardDiff.Dual's implementation is more complicated than DualNumber.Dual. IMO, the extra complexity is beyond worth it for the really useful features you gain, e.g. multidimensional dual numbers and the tagging system (enabling, for example, safe nesting of dual numbers).
I can't be sure about breaking changes until we actually make the switch, but there shouldn't be any actual loss in functionality.
Risking sounding like a broken record, dual numbers extend their underlying field. From an abstract view, it seems unreasonable to subtype them as reals.
This has been discussed at significant length; one of the primary discussions is here, and there are a couple of other discussions scattered around if you want to hunt them down.
My view is that Julia's type hierarchy establishes the subtype relations which drive dispatch. Borrowing naming conventions from math, while useful for driving design, does not guarantee (or even necessarily imply) that this type hierarchy will obey external formalisms.
For AD purposes, Dual <: Real is the right choice because it is more programmatically practical than Dual <: Number for interface implementation and injectability. In the >1 year since making that change in ForwardDiff, we haven't run into any notable problems because of it, and I think it's turned out to be the right decision.
@mikaelslevinsky I think the idea is that only Complex{<:Dual} would be allowed, no longer Dual{<:Complex}. I personally don't see any downsides.
Another benefit of the ForwardDiff.Dual subtyping as Real is that other packages that want to override uses of Dual and Complex don't have to recursively subtype as:
immutable MyExtension{T <: Union{Real, Complex, Dual}} <: Number
fields::T
end
That would be endless.