feature/analyzers - symbolic shape checking

[dsyme]

A bunch of additional work over #207 to host shape checking tooling as an F# analyzer rather than as a standalone tool

Requires

• dotnet/fsharp branch feature/analyzers
• fsprojects/FSharp.Compiler.PortaCode branch feature/analyzers
• DiffSharp branch feature/analyzers (this PR) or feature/merged which includes this branch

Runs live checking as an F# analyzer

1. Parallel build dotnet/fsharp branch feature/analyzers

   .\build -pack -c Debug
   .\build -deploy -c Release
   

2. Parallel build FSharp.Compiler.Portacode branch feature/analyzers

   dotnet build
   dotnet publish
   

3. Parallel build DiffSharp branch feature/merged which includes this branch

   dotnet build 
   

Step by step manual testing

This builds on a stack of things in the pipeline so careful debugging is needed as things go awry. The following test the tech step by step

• [x] netfx fsc + Test.Analyzer.dll (two warnings should appear)

c:\GitHub\dsyme\fsharp\artifacts\bin\fsc\Debug\net472\fsc.exe --compilertool:C:\GitHub\dsyme\fsharp\artifacts\bin\Test.Analyzer\Debug\netstandard2.0 C:\misc\neutral-script.fsx

• [x] netcore fsc.exe + Test.Analyzer.dll (two warnings should appear)

dotnet c:\GitHub\dsyme\fsharp\artifacts\bin\fsc\Debug\net5.0\fsc.dll --compilertool:C:\GitHub\dsyme\fsharp\artifacts\bin\Test.Analyzer\Debug\netstandard2.0 C:\misc\neutral-script.fsx

• [x] netcore fsi.exe + Test.Analyzer.dll ('couldn't find stdin' should appear - correct because analyzer does this)

dotnet c:\GitHub\dsyme\fsharp\artifacts\bin\fsi\Debug\net5.0\fsi.dll --runanalyzers --compilertool:C:\GitHub\dsyme\fsharp\artifacts\bin\Test.Analyzer\Debug\netstandard2.0 C:\misc\neutral-script.fsx

• [x] netfx fsc + FSharp.Tools.LiveChecks.Analyzer sample (four errors should appear)

c:\GitHub\dsyme\fsharp\artifacts\bin\fsc\Debug\net472\fsc.exe --compilertool:e:\GitHub\dsyme\FSharp.Compiler.PortaCode\FSharp.Tools.LiveChecks.Analyzer\bin\Debug\netstandard2.0 e:\GitHub\dsyme\FSharp.Compiler.PortaCode\tests\FSharp.Tools.LiveChecks.Tests\test.fsx

• [x] netcore fsi + FSharp.Tools.LiveChecks.Analyzer sample - one error should appear (due to stop on first error) - note compiler too is via #compilertool in script

dotnet c:\GitHub\dsyme\fsharp\artifacts\bin\fsi\Debug\net5.0\fsi.dll  --runanalyzers  e:\GitHub\dsyme\FSharp.Compiler.PortaCode\tests\FSharp.Tools.LiveChecks.Tests\test.fsx

• [x] netcore fsi + FSharp.Tools.LiveChecks.Analyzer + DiffSharp ShapeCheck with --typecheckonly --runanalyzers

dotnet c:\GitHub\dsyme\fsharp\artifacts\bin\fsi\Debug\net5.0\fsi.dll --typecheckonly --runanalyzers E:\GitHub\dsyme\DiffSharp\examples\vae.fsx

• [ ] Smoke test to put in script for execution while editing:

#compilertool @"e:\GitHub\dsyme\FSharp.Compiler.PortaCode\FSharp.Tools.LiveChecks.Analyzer\bin\Debug\netstandard2.0"

• [ ] Eventually this to get a sample analyzer via the nuget package:

#r "nuget: FSharp.Tools.LiveChecks.Analyzer"

Requires branch feature/analyzers from dotnet/fsharp

• the actual fslive.exe tool is run as a subsidiary process regardless of the environment of the tool the analyzer is running in. Inputs are binary-serialized back and forth
• prototype is windows only
• no recovery for subsidiary process
• no adjustment is referenced DLLs for subsidiary process change
• subsidiary processes may currently leak (cleanup on exit of host)

Using with DiffSHarp

c:\GitHub\dsyme\fsharp\artifacts\bin\fsc\Debug\net472\fsc.exe --compilertool:e:\GitHub\dsyme\FSharp.Compiler.PortaCode\FSharp.Tools.LiveChecks\bin\Debug\netstandard2.0\FSharp.Tools.LiveChecks.dll E:\GitHub\dsyme\DiffSharp\examples\vae.fsx

Won't be usable for a while, expect this to stay open a long time

[codecov[bot]]

Codecov Report

:exclamation: No coverage uploaded for pull request base (feature/sym@1920c8e). Click here to learn what that means. The diff coverage is n/a.

@@              Coverage Diff               @@
##             feature/sym     #274   +/-   ##
==============================================
  Coverage               ?   58.97%           
==============================================
  Files                  ?       36           
  Lines                  ?     7830           
  Branches               ?     1777           
==============================================
  Hits                   ?     4617           
  Misses                 ?     2330           
  Partials               ?      883           

[pkese]

It might be interesting to extend this also with:

1. some sort of checking of implicit conversion of dtype of tensor
   In Torch I sometimes get surprised by the fact that the algorithm is running slowly only to find out that I someplace multiplied a float32 tensor with a float64 and then Torch silently renders float64 computations.

2. knowing which tensor is on which device
   This is a bit lesser problem in practice, because you get proper feedback in form of a runtime error, but it would nevertheless be a boon to see this while editing (similarly as we have analyzers for mismatching shapes even though that also pops up as runtime errors).

[dsyme]

Agreed, both are simple.

I think I will also extend this with a standalone command line tool to allow it all to be run without the F# analyser feature in place All with the aim of trialling it earlier