Flux.jl icon indicating copy to clipboard operation
Flux.jl copied to clipboard
verified publisher icon FluxML

Create a getting started section and add a new linear regression example

Open Saransh-cpp opened this issue 3 years ago • 19 comments

This PR -

  • Adds a new "Getting Started" section in the docs which I will be working on for the next couple of weeks
  • Moves basics.md and overview.md to the "Getting Started" section. Currently, there are no changes in the files, but they will be changed soon.
  • Adds a Linear Regression example to the "Getting Started" section

For the linear regression guide -

  • Should I add the backpropagation step in the first half, or should I leave it to the Flux.Optimise.update! step?
  • This guide overlaps with a lot of other textual information present but scattered in Flux's docs. These other texts will also be updated or moved to a better place soon. Basics.md, for example, does something very similar with dummy data, and the current "Getting Started" guide does something similar but with pre-defined weights.

PR Checklist

  • [ ] Tests are added
  • [ ] Entry in NEWS.md
  • [ ] Documentation, if applicable

Saransh-cpp avatar Jul 05 '22 20:07 Saransh-cpp

Should we consider having a "getting started" on the website in addition to the docs?

On Wed, Jul 6, 2022, 01:53 Saransh @.***> wrote:

This PR -

  • Adds a new "Getting Started" section in the docs which I will be working on for the next couple of weeks
  • Moves basics.md and overview.md to the "Getting Started" section. Currently, there are no changes in the files, but they will be changed soon.
  • Adds a Linear Regression example to the "Getting Started" section

For the linear regression guide -

  • Should I add the backpropagation step in the first half, or should I leave it to the Flux.Optimise.update! step?
  • This guide overlaps with a lot of other textual information present but scattered in Flux's docs. These other texts will also be updated or moved to a better place soon. Basics.md, for example, does something very similar with dummy data, and the current "Getting Started" guide does something similar but with pre-defined weights.

PR Checklist

  • Tests are added
  • Entry in NEWS.md
  • Documentation, if applicable

You can view, comment on, or merge this pull request online at:

https://github.com/FluxML/Flux.jl/pull/2016 Commit Summary

File Changes

(9 files https://github.com/FluxML/Flux.jl/pull/2016/files)

Patch Links:

  • https://github.com/FluxML/Flux.jl/pull/2016.patch
  • https://github.com/FluxML/Flux.jl/pull/2016.diff

— Reply to this email directly, view it on GitHub https://github.com/FluxML/Flux.jl/pull/2016, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJOZVVPWCTKUTEJDXS5J7ILVSSKTHANCNFSM52XLUU3A . You are receiving this because you are subscribed to this thread.Message ID: @.***>

DhairyaLGandhi avatar Jul 05 '22 21:07 DhairyaLGandhi

The first thought I had was to add a hyperlink on the website's Getting Started page, which would redirect the user to the docs' Getting Started page (similar to the Ecosystem page).

Should I also add the same tutorial (once approved) to the website, or should we link the section?

Saransh-cpp avatar Jul 06 '22 10:07 Saransh-cpp

I think there is a consensus to not use Flux.params API any longer. We should not introduce it into new tutorials being written.

avik-pal avatar Jul 07 '22 04:07 avik-pal

Thank you for the suggestion! I took some time to go through Optimisers.jl, and I am assuming that the traditional Flux training method should be replaced with Optimisers.jl?

Saransh-cpp avatar Jul 08 '22 16:07 Saransh-cpp

Agree we should move away from the whole weird Params story.

But perhaps this linear regression example should delay introducing Optimisers.jl as long as possible. For simple gradient descent, just writing out something like this:

dLdW, _, _ = gradient(loss, W, x, y)
W .= W .- 0.1 .* dLdW

might be a better level than immediately introducing optimiser state etc. It's unfortunate that this has quite a few moving parts, unlike train!'s apparent simplicity (although really train! hides a lot & this is also confusing). Maybe Optimisers.jl ought to be introduced along with an explanation that adding momentum helps, so that you know why there is a state?

Even something like this seems OK to me, pretty explicit, and makes you understand why you are about to see a tool for walking over the arrays:

m = Dense(1 => 1)
for step in 1:10
  dLdm, _, _ = gradient(loss, m, x, y)
  
  m.weight .= m.weight .- 0.1 .* dLdm.weight
  m.bias .= m.bias .- 0.1 .* dLdm.bias
end

One more comment. In such tutorials, things like params = Flux.params(W, b) seem super-confusing. It would be nice to choose variable names which are very clearly things you've chosen, not features of Flux. flux_model is good.

(Link to rendered version: https://github.com/Saransh-cpp/Flux.jl/blob/linear-regression/docs/src/getting_started/linear_regression.md )

mcabbott avatar Jul 13 '22 16:07 mcabbott

Agreed, we should keep the linear regression example simple. I will update the tutorial to show the gradient descent algorithm in action.

I have been working on the logistic regression example locally and will update that with the same.

Saransh-cpp avatar Jul 14 '22 11:07 Saransh-cpp

I'll update this with the new train! definition once #2029 is merged. Right now this PR does not use train! in any way.

Saransh-cpp avatar Jul 31 '22 18:07 Saransh-cpp

Test: @ModelZookeeper commands

Saransh-cpp avatar Aug 07 '22 13:08 Saransh-cpp

Thanks for the review, @mcabbott! Pluto sounds good! We can get rid of the copy-paste section if this is converted to a Pluto notebook (or file, not sure, will go through it in detail).

(SciML uses this copy-paste section at the top, but this code was too lengthy to be placed at the top)

Edit: A discussion about the documentation of Metalhead is going on at https://github.com/FluxML/Metalhead.jl/pull/199, which could result in a uniform template for these getting started/quickstart guides.

Saransh-cpp avatar Aug 15 '22 20:08 Saransh-cpp

@mcabbott, JuliaManifolds/Manopt.jl renders Pluto notebooks in the Documenter documentation using the following make.jl contents - https://github.com/JuliaManifolds/Manopt.jl/blob/master/docs/make.jl#L1-L106.

This does look a bit hacky, and it also distorts the documentation when a user opens one of the rendered Pluto notebooks -

Sidebar

Normal page

image

Rendered Pluto notebook

image

Settings

Normal page

image

Rendered Pluto notebook

image

Is there a legitimate way to render Pluto notebooks in Documenter's documentation? This hack does not look good to me. Alternatively, we could keep the section as it is and at the top link a Pluto notebook that can be downloaded for following along.

Here is how the converted Pluto notebook looks like - https://saransh-cpp.github.io/assets/pluto/linear_regression.jl.html

Note: I am just redirecting users to the html page generated by Pluto from here - https://saransh-cpp.github.io/blog/ (this will be removed once this PR is merged to avoid duplicate pages on the web).

Saransh-cpp avatar Aug 21 '22 10:08 Saransh-cpp

Currently, https://fluxml.ai/Flux.jl/stable/models/overview/ does something similar but is not as extensive as this guide. Should it be removed, or should it also be put under the "Getting Started" guide with a better title?

Saransh-cpp avatar Aug 22 '22 17:08 Saransh-cpp

What if anything do we lose if it's removed? I think it would be nice to do so, but any material in it which isn't covered elsewhere would need a new home.

ToucheSir avatar Aug 23 '22 01:08 ToucheSir

I think the following text can be used at the top of this guide. The rest of this page is definitely a subset of this guide.

Flux is a pure Julia ML stack that allows you to build predictive models. Here are the steps for a typical Flux program:

  • Provide training and test data
  • Build a model with configurable parameters to make predictions
  • Iteratively train the model by tweaking the parameters to improve predictions
  • Verify your model

Under the hood, Flux uses a technique called automatic differentiation to take gradients that help improve predictions. Flux is also fully written in Julia so you can easily replace any layer of Flux with your own code to improve your understanding or satisfy special requirements.

Here's how you'd use Flux to build and train the most basic of models, step by step.

Saransh-cpp avatar Aug 23 '22 14:08 Saransh-cpp

Currently, https://fluxml.ai/Flux.jl/stable/models/overview/ does something similar but is not as extensive as this guide. Should it be removed, or

One virtue of that page is that it's much shorter.

I like this PR's story, it's a nice ground-up explanation. But if you have met some of this before, and want a faster path to seeing how to write it in Julia's syntax and using Flux's pieces, then perhaps you prefer the older one.

Not sure what the ideal way to organise this material is...

mcabbott avatar Aug 27 '22 18:08 mcabbott

But if you have met some of this before, and want a faster path to seeing how to write it in Julia's syntax and using Flux's pieces, then perhaps you prefer the older one.

Yes, this makes sense. Maybe converting it into a "Quickstart" page under the "Getting Started" section?

Saransh-cpp avatar Aug 29 '22 17:08 Saransh-cpp

In addition to the main manual

https://fluxml.ai/Flux.jl/stable/models/overview/

we also have some quite nice tutorials here:

https://fluxml.ai/tutorials.html

How do we make these all findable, and where does this new page go?

The main docs also seem a slightly awkward combination of introduction and reference. It's possible that the "Building models" heading should be split in two? Not sure.

Screenshot 2022-08-29 at 15 12 58

mcabbott avatar Aug 29 '22 19:08 mcabbott

In addition to the main manual

https://fluxml.ai/Flux.jl/stable/models/overview/

we also have some quite nice tutorials here:

https://fluxml.ai/tutorials.html

How do we make these all findable, and where does this new page go?

While drafting the "Getting Started" section, I wanted to include only the guides that will get a user started with flux. The website tutorials should be the ones introducing something that a user doesn't find themselves engaged with when they begin with ML/DL or an ML/DL package, for example, GANs and Transfer Learning. I think the DataLoader tutorial should be moved to the MLUtils page and the Deep Learning with Flux - A 60 Minute Blitz (September 2020) can be added to the "Getting Started" section.

Ideally, there should be a "Tutorials" heading on the docs' sidebar which should redirect users to the website's tutorials page. Similarly, the website's getting started page should redirect users to the docs' getting started section.

A nice infographic -

image

IMO, the DataLoader example is a "How-To Guide", the "Getting Started" guides are more inclined towards "Explanations", and the website tutorials are "Tutorials".

Saransh-cpp avatar Sep 01 '22 16:09 Saransh-cpp

I was looking at Turing's documentation a bit. They clearly face the same problem, that there are many levels at which you can introduce something.

One thing I do like is that they have a compact example which basically does what the library does, up front, in one code block. Anyone can copy and run this in one go. Experts who know another package / language can see roughly what the notation is etc. Beginners will be mystified as to how it's working, but at least you get a plot which shows something worked.

My first attempt at such an example is here: https://gist.github.com/mcabbott/a61930793c5a064e10411ec427a6377a . Whether this ought to be instead of, or in addition to, existing text I don't know.

mcabbott avatar Sep 19 '22 14:09 mcabbott

One thing I do like is that they have a compact example which basically does what the library does, up front, in one code block. Anyone can copy and run this in one go. Experts who know another package / language can see roughly what the notation is etc. Beginners will be mystified as to how it's working, but at least you get a plot which shows something worked.

Yes! These are the how-to guides. I find them extremely helpful as well!

I am not sure where these small tutorials should be added?

Saransh-cpp avatar Sep 22 '22 04:09 Saransh-cpp

Hmm, where should this tutorial go now? We do have a new Getting Started section, so should this go in there (along with other tutorials)?

Saransh-cpp avatar Oct 18 '22 14:10 Saransh-cpp

Good question. PyTorch and JAX have tutorial sections for this, but at present we've outsourced that to the site. Maybe we could add one and pull a couple of website tutorials in for a later PR (but link to them for now)? That would also help prevent bitrot of the tutorials on the site.

ToucheSir avatar Oct 21 '22 04:10 ToucheSir

So would that mean shifting the tutorials section to the Documenter website completely? Maybe I can shift the existing section before adding this guide to the same. We could still keep the website navbar link, but instead, link it to the new section in the Documenter site (like the ecosystem page)?

Saransh-cpp avatar Oct 24 '22 17:10 Saransh-cpp

I like that idea. It might be easier to create that section in this PR and do a follow-up for migrating the other tutorials.

ToucheSir avatar Oct 24 '22 19:10 ToucheSir

This PR now creates a new "tutorials" section. I will migrate the website's tutorials here in the follow-up PRs.

Saransh-cpp avatar Oct 25 '22 18:10 Saransh-cpp

Note that there's already a tutorials section (on master) so simplest to add this there?

mcabbott avatar Oct 25 '22 19:10 mcabbott

Note that there's already a tutorials section (on master) so simplest to add this there?

Ah, I missed it because of how low it is in the sidebar. Maybe we should move it above the "Performance Tips" section?

Saransh-cpp avatar Oct 27 '22 09:10 Saransh-cpp