Changes to Docs

[willtebbutt]

Lots has changed since the docs were first written. #152 addresses a number of things, but there are a few more things that we might want to consider:

• changing all references to autodiff / automatic differentiation to AD / algorithmic differentiation, with a terminology box in the docs somewhere, explaining what we're on about.
• In the "On writing good rrule and frule " bit, we should consider modifying the recommendations for Zero() or One(). In particular, removing any mention of One because it's not generally appropriate or helpful. Additionally, the "Write Tests" section should probably be entitled "Write Tests using FiniteDifferences". Similarly, the "CAS Systems are your friends" should have some reference to them not being helpful when writing tests for derivatives.

Additionally, the section of the docs on Differentials needs to be expanded / modified to make the following points:

• is the only operation guaranteed to be defined on differentials and, moreover, it's only guaranteed to be defined between differentials that are valid for the same primal type.

• is only guaranteed to be defined between scalars and differentials. You can't generally multiple a differential by another differential

• between primals and differentials isn't always defined, but it is for a lot of interesting cases e.g. Real, Matrix{<:Real} etc stuff that I've missed that we've figured out since this section was last written.

[nickrobinson251]

why the "automatic" -> "algorithmic" change. I think they refer to the same thing, but with "auto" the more dominant terminology. This is what wikipedia implies too https://en.wikipedia.org/wiki/Automatic_differentiation

[willtebbutt]

Yeah, I agree that it's the more dominant, but Griewank and Walther prefer algorithmic. In my opinion "automatic differentiation" sounds way to magical. I can live with automatic, but would rather not.