Moving (parts of) documentation generation scripts to the compiled dll

[Krzysztof-Cieslak]

This was suggested by @dsyme yesterday on Twitter.

Core Design:

• As Fornax is essentially just an opinionated script runner there is no way (by-design) to remove .fsx scripts completely.
• We could potentially move parts of the scripts (for example API reference generation etc) to compiled DLLs - and our generators/loaders .fsx would just contain a reference to this DLL and single one-line function call

Pros:

• Easy to update to a new version (just run paket update on the Docs group)
• Hides internal API changes (for example users doesn't need to know that F#.Formatting API has changed)
• Provides a single opinionated way of how documentation should be build - good for consistency across our OSS ecosystem

Cons:

• Provides a single opinionated way of how documentation should be build - different projects may have different needs
• Lack of customizability - a huge advantage of having those scripts in the repo is the fact you can just easily change them to make them fit project needs better
• Distribution story when not using Paket - since plain NuGet nowadays installs packages globally there won't be an easy way to reference this library from our generators/loaders scripts (Maybe solved by #r nuget - but keep in mind it's still a preview feature)

CC: @baronfel

[dsyme]

Just playing around with this in the DiffSharp repo based on cutting and pasting things from SampleWaypoint. Random unreasonable wish list below :-))

1. the checkin of lib/Fornax.Core.dll feels off. Hoping not to need that.

2. I'd like to use it without Paket or FAKE, we're not using those in the DiffSharp repo.

3. I think my ideal user experience would be dotnet fornax build (or dotnet waypoint build) in the top root directory and that's all. No other installation needed besides the tool. The ideal result of this would be

   • API docs for all packable DLLs generated
   • Page/tutorial docs for all .md and .fsx under docsrc\... built
   • All output in docs directory ready to checkin, or dotnet fornax build -o ..\tmp\DiffSharp-gh-pages to dump them into another copy of the repo with gh-pages checked out
   • No generation scripts checked in at all

   I understand how hard it is to get this, but I just wanted to record that. I think if we achieved it we'd use that in every fsprojects repo

[dsyme]

OK I managed to generate some docs for DiffSharp.

There are a lot of good things here. As an F# repo maintainer my biggest concern about using the tool is definitely the 6MB of stuff to checkin under docs and the long term maintenance of that under iterative updates and improvements to Fornax/Waypoint. I'd rather checkin basically nothing. Any chance that can all be bundled in the Fornax tool or something?

I'd have a whole lot of improvements to make for the generated docs, e.g. some initial thoughts are:

1. Don't show signatures. Instead show usage, e.g. instead of

   GetSlice
   Signature: (i0:int * i1:int) -> Tensor
   Attributes: ExcludeFromCodeCoverageAttribute
   

   show a typical callsite something like this

   tensor.GetSlice(i0, i1)       | Summary text
                                 | 
                                 | i0: int  The first index
                                 | i1: int  The second index
                                 | returns: Tensor
   

   etc. Don't show the attributes.

2. There are extra parentheses being show in signatures

I'd be happy to help make such changes and simplifications.

[Krzysztof-Cieslak]

the checkin of lib/Fornax.Core.dll feels off. Hoping not to need that.

This will be fixed with #r nuget, hopefully pretty soon

I'd like to use it without Paket or FAKE, we're not using those in the DiffSharp repo.

I think #r nuget will remove need for Paket, and FAKE is not needed for documentation generation part. Waypoint is more than just documentation generation (it's trying to fill the same niche as ProjectScaffold did in the past with build process, release notes, CI etc), but if we talk just about documentation part of it then FAKE is not needed - it's basically dotnet fornax build inside of the docs folder

I think my ideal user experience would be ...

Personally, I agree. Heck, I'd move even further than this - docs should just be hosted in a service, you just submit your package and content files - similarly to how HexDocs works for Elixir...

But again it's an issue of the customizability - what if you want to add more links in the menu. Or what if you don't like menu structure (fairly opinionated) that I'm providing? By giving scripts we enable people to change things people don't like

[Krzysztof-Cieslak]

6MB of stuff to checkin under

It seems 5MB of this are static files we provide. Probably most of it could be replaced with the calls to CDNs rather than having it in the repo. I'll investigate. Another 700KBs is Fornax.Core.dll and yes, we need to get rid of it.

[dsyme]

Yup

I guess ideally I'd only want things under docs if it's actual customization (logos, names, content). If it's just default behaviour then just let the defaults be in the DLL.

I don't have much experience with how other languages are doing this.

[Krzysztof-Cieslak]

To reiterate on ideal UX point - from my experience in building and trying to push tooling in our ecosystem it seems that really often our users have their own preferred workflows - we always have these discussions of Paket vs NuGet, or about how FAKE should be used, or about how editor tooling should work, or what should be part of solution scaffold, or how we should create projects. Such discussions happened with Paket, FAKE, ProjectScaffold, Ionide, Forge, even SAFE Stack template... any basically any other tooling project that's out there.

It just seems that our community strongly values their independence of choice in the area of tooling/workflow - not saying it's good or bad, it's just what it is. So I'm not sure if such opinionated tool as you suggest Don would work in F# context - I've been trying to push similar opinionated tools before and it was not the most successful thing I've done in my life ;-)

[dsyme]

It just seems that our community strongly values their independence of choice in the area of tooling/workflow - not saying it's good or bad, it's just what it is

Historically doc generation for most "fsprojects" projects tended to not be quite like this - people took what they got in the ProjectScaffold template and made lots of sites that look like this or this. The look is good and I would be happy with that (or a visual refresh of that) on .NET Core tbh. The API reference doc generation used in that generation needed more work though

So I think there's space for at least one opinionated tool, at least for the majority of fsprojects/... projects, but there are requirements, e.g. content driven, clean simple look, a few necessary customization hooks.

[Krzysztof-Cieslak]

people took what they got in the ProjectScaffold template and made lots of sites that look like this or this.

Just for the sake of adding to the discussion about the earlier point about not committing generator .fsx scripts - this was not a case with ProjectScaffold. Those projects have not only F# scripts in there (https://github.com/fsharp/FSharp.Data/blob/master/docs/tools/generate.fsx) but also Razor templates (https://github.com/fsharp/FSharp.Data/blob/master/docs/tools/templates/template.cshtml) in the repo - in case of Fornax we've replaced Razor with more .fsx scripts ;-)

The API reference doc generation used in that generation needed more work though

Definitely agree that improvements need to be made - I'm still actively working on making this documentation template better (using Waypoint repository as a current source of truth for the documentation template)

[dsyme]

Those projects have not only F# scripts in there (https://github.com/fsharp/FSharp.Data/blob/master/docs/tools/generate.fsx) but also Razor templates (https://github.com/fsharp/FSharp.Data/blob/master/docs/tools/templates/template.cshtml) in the repo - in case of Fornax we've replaced Razor with more .fsx scripts ;-)

Yes, I know, and it's my experience with maintaining these that make me want to never want to see them again :-) They were continually breaking in very obscure ways, and have now ultimately bit-rotted completely unforunately.

The scripting/dependency experience will be more uniform now with .NET Core FSI scripting and #r "nuget: ...". But boy those scripts have caused me a lot of pain back a few years ago.

[NatElkins]

@Krzysztof-Cieslak Just want to chime in with my thoughts on user-friendliness. I also agree that having a compiled DLL with common helper functions and a discoverable API would improve things. Both in Waypoint and fornax new it can be a little hard to get your bearings because it's all "just code." For example when you do dotnet new Waypoint or fornax new, trying to understand all the code in lunr.fsx or apiref.fsx or postloader.fsx can be pretty overwhelming. I think it would be easier to understand what is going on if the common logic were hidden away behind a function. And if a user needs to customize, they can always just write their own custom F# code, since as you said Fornax is basically a script runner.

[Krzysztof-Cieslak]

6MB of stuff to checkin under

With #31 this goes down from 6MB to 900KB - and 700KB out of it is still Fornax.Core.dll that I hope to remove with #r nuget... thingy.