Building and testing with RFCs FS-1043 FS-1071

[dsyme]

I'm opening this issue to cover steps to build and test this library with the implementations of RFCs FS-1043 FS-1071 activated

https://github.com/dotnet/fsharp/pull/6805

https://github.com/dotnet/fsharp/pull/6810

We can also look at adjusting the library to make use ot the features

[dsyme]

I'm currently testing by setting these

  <PropertyGroup>
    <FscToolPath>c:\GitHub\dsyme\fsharp2\artifacts\bin\fsc\Debug\net472</FscToolPath>
    <FscToolExe>fsc.exe</FscToolExe>
  </PropertyGroup>

I will also test with

  <PropertyGroup>
    <OtherFlags>--langversion:preview</OtherFlags>
  </PropertyGroup>

[dsyme]

Separately, I notice compilation is quite slow, I presume this is related to SRTP constraint processing (I'm also using the debug compiler). It would be wise to do a fulll compiler profiling run both before and after

[smoothdeveloper]

@dsyme, just FYI, in current compiler, there is an unreported compiler bug that makes buiding the F#+ test project in DEBUG not possible, you need to build that test project in RELEASE.

[dsyme]

@dsyme, just FYI, in current compiler, there is an unreported compiler bug that makes buiding the F#+ test project in DEBUG not possible, you need to build that test project in RELEASE.

OK, I'm just using .\build

Could we get that bug reported? Even if it'snot fixed that's the sort of corner case problem I'd really like to get pinned down with testing before we proceed with the RFCs

[dsyme]

OK turning on /langversion:preview with RFC 6805 does indeed cause a compilation error. I'll look at it now

     2>C:\GitHub\dsyme\FSharpPlus\src\FSharpPlus\Data\Seq.fs(11,87): error FS0043: A unique overload for m
       ethod 'Sequence' could not be determined based on type information prior to this program point. A t
       ype annotation may be needed. Candidates: static member Async.Sequence : t:seq<Async<'a>> -> Async<
       seq<'a>>, static member Option.Sequence : t:seq<'T option> -> seq<'T> option [C:\GitHub\dsyme\FShar
       pPlus\src\FSharpPlus\FSharpPlus.fsproj]

     2>C:\GitHub\dsyme\FSharpPlus\src\FSharpPlus\Data\Seq.fs(15,67): error FS0043: A unique overload for m
       ethod 'Sequence' could not be determined based on type information prior to this program point. A t
       ype annotation may be needed. Candidates: static member Async.Sequence : t:seq<Async<'a>> -> Async<
       seq<'a>>, static member Option.Sequence : t:seq<'T option> -> seq<'T> option [C:\GitHub\dsyme\FShar
       pPlus\src\FSharpPlus\FSharpPlus.fsproj]
     2>CoreCompile:

     2>C:\GitHub\dsyme\FSharpPlus\src\FSharpPlus\Data\Seq.fs(11,87): error FS0043: A unique overload for m
       ethod 'Sequence' could not be determined based on type information prior to this program point. A t
       ype annotation may be needed. Candidates: static member Async.Sequence : t:seq<Async<'a>> -> Async<
       seq<'a>>, static member Option.Sequence : t:seq<'T option> -> seq<'T> option [C:\GitHub\dsyme\FShar
       pPlus\src\FSharpPlus\FSharpPlus.fsproj]

     2>C:\GitHub\dsyme\FSharpPlus\src\FSharpPlus\Data\Seq.fs(15,67): error FS0043: A unique overload for m
       ethod 'Sequence' could not be determined based on type information prior to this program point. A t
       ype annotation may be needed. Candidates: static member Async.Sequence : t:seq<Async<'a>> -> Async<
       seq<'a>>, static member Option.Sequence : t:seq<'T option> -> seq<'T> option [C:\GitHub\dsyme\FShar
       pPlus\src\FSharpPlus\FSharpPlus.fsproj]

[gusty]

It's actually a very long longstanding bug, I never managed to find the time to get mini-repro. That's why I never reported it in the FSharp compiler.

[gusty]

That error you're getting regarding Aync.Sequence might be because now that extension is built-in with a slightly different signature.

So it's not related to your work.

[dsyme]

So it's not related to your work.

I tried changing the spelling of Sequence to Seqquence throughout FSHarpPlus and I still get this new error. And it only happens with /langversion:preview. So it's a real change in behaviour.

Note there is definitely a difference in logic that could somehow explain this because the extension members like Option.Seqquence and Async.Seqquence are being taken into account where they wouldn't have been before. Interesting.

[dsyme]

I managed to fix the bug found above, and FSHarpPlus now compiles with the preview of FS-1043

However

1. the tests failto compile
2. the compilation seems slower (there's a possible reason for that, and I'm using the Debug compiler, but still it seems much slower)

I will look at these tomorrow

The test compilation failure is this:

  C:\GitHub\dsyme\FSharpPlus\src\FSharpPlus\Samples\Learn You a Haskell.fsx(36,13): error FS0073: internal error: Undefined or unsolved type variable:  ^_?319442Stack TraceFSharp.Compiler.ErrorLogger+InternalError: Undefined or unsolved type variable:  ^_?319442C:\GitHub\dsyme\FSharpPlus\src\FSharpPlus\Samples\Learn You a Haskell.fsx (36,12--36,50) IsSynthetic=false [C:\GitHub\dsyme\FSharpPlus\src\FSharpPlus.Samples\FSharpPlus.Samples.fsproj]

[gusty]

Just to clarify, that's not really from the tests. It comes from the samples folder which has a project. Though those samples don't have any test at all, we did a project to make sure they keep compiling. Thanks for looking at this.

[dsyme]

1. I checked this with the release compiler and compilation is no slower without --langversion:preview activated

2. We now get a new error for --langversion:preview. I must say that the contents of the error message make it look like a complete nightmare :)

warning FS0075: The command-line option 'times' is for test purposes only

C:\GitHub\dsyme\FSharpPlus\src\FSharpPlus\Data\Cont.fs(39,59): warning FS1125: The instantiation of the generic type 'Cont' is missing and can't be inferred from the arguments or return type of this member. Consider providing a type instantiation when accessing this type, e.g. 'Cont<_,_>'.

C:\GitHub\dsyme\FSharpPlus\src\FSharpPlus\Data\ZipList.fs(42,65): error FS0043: No overloads match for method 'Map'. The available overloads are shown below.
Possible overload: 'static member Map.Map : ( ^Profunctor<'B,'C> * ('C -> 'D)) * _mthd:Internals.Default5 -> 'Profunctor<'B,'D> when  ^Profunctor<'B,'C> : (static member Dimap :  ^Profunctor<'B,'C> * ('a1 -> 'a1) * ('C -> 'D) -> 'Profunctor<'B,'D>)'. Type constraint mismatch. The type     'ZipList<'a> * ('g -> 'g -> 'g)'    is not compatible with type    ''b * ('d -> 'e)'    .
Possible overload: 'static member Map.Map : ( ^Bifunctor<'T,'V> * ('V -> 'W)) * _mthd:Internals.Default6 -> 'a4 when  ^Bifunctor<'T,'V> : (static member Bimap :  ^Bifunctor<'T,'V> * ('a1 -> 'a1) * ('V -> 'W) -> 'a4)'. Type constraint mismatch. The type     'ZipList<'a> * ('g -> 'g -> 'g)'    is not compatible with type    ''b * ('d -> 'e)'    .
Possible overload: 'static member Map.Map : ( ^t * 'a1) * _mthd:Internals.Default1 -> unit when  ^t : null and  ^t : struct'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    ''b * 'c'    .
Possible overload: 'static member Map.Map : ( ^Functor<'T> * ('T -> 'U)) * _mthd:Internals.Default1 -> 'Functor<'U> when  ^Functor<'T> : (static member Map :  ^Functor<'T> * ('T -> 'U) -> 'Functor<'U>)'. Type constraint mismatch. The type     'ZipList<'a> * ('f -> 'f -> 'f)'    is not compatible with type    ''b * ('c -> 'd)'    .
Possible overload: 'static member Map.Map : (System.IObservable<'T> * ('T -> 'U)) * _mthd:Internals.Default2 -> System.IObservable<'U>'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    'System.IObservable<'b> * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : (System.Collections.Generic.IReadOnlyDictionary<'Key,'T> * ('T -> 'U)) * _mthd:Internals.Default2 -> System.Collections.Generic.IReadOnlyDictionary<'Key,'U> when 'Key : equality'. Type constraint mismatch. The type     'ZipList<'a> * ('e -> 'e -> 'e)'    is not compatible with type    'System.Collections.Generic.IReadOnlyDictionary<'b,'c> * ('c -> 'd)'    .
Possible overload: 'static member Map.Map : (System.Collections.Generic.IDictionary<'Key,'T> * ('T -> 'U)) * _mthd:Internals.Default2 -> System.Collections.Generic.IDictionary<'Key,'U> when 'Key : equality'. Type constraint mismatch. The type     'ZipList<'a> * ('e -> 'e -> 'e)'    is not compatible with type    'System.Collections.Generic.IDictionary<'b,'c> * ('c -> 'd)'    .
Possible overload: 'static member Map.Map : (System.Collections.Generic.IEnumerator<'T> * ('T -> 'U)) * _mthd:Internals.Default2 -> System.Collections.Generic.IEnumerator<'U>'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    'System.Collections.Generic.IEnumerator<'b> * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : (seq<'T> * ('T -> 'U)) * _mthd:Internals.Default2 -> seq<'U>'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    'seq<'b> * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : ( ^Applicative<'T> * ('T -> 'U)) * _mthd:Internals.Default3 ->  ^Applicative<'U> when  ^Applicative<'T> : (static member ( <*> ) :  ^Applicative<'T->'U> *  ^Applicative<'T> ->  ^Applicative<'U>) and ( ^Applicative<'T->'U> or  ^Applicative<'T> or  ^Applicative<'U>) : (static member ( <*> ) :  ^Applicative<'T->'U> *  ^Applicative<'T> ->  ^Applicative<'U>) and  ^Applicative<'T->'U> : (static member Return : ('T -> 'U) ->  ^Applicative<'T->'U>)'. Type constraint mismatch. The type     'ZipList<'a> * ('g -> 'g -> 'g)'    is not compatible with type    ''b * ('d -> 'e)'    .
Possible overload: 'static member Map.Map : ( ^Monad<'T> * ('T -> 'U)) * _mthd:Internals.Default4 ->  ^Monad<'U> when  ^Monad<'T> : (static member ( >>= ) :  ^Monad<'T> * ('T ->  ^Monad<'U>) ->  ^Monad<'U>) and ( ^Monad<'T> or  ^Monad<'U>) : (static member ( >>= ) :  ^Monad<'T> * ('T ->  ^Monad<'U>) ->  ^Monad<'U>) and  ^Monad<'U> : (static member Return : 'U ->  ^Monad<'U>)'. Type constraint mismatch. The type     'ZipList<'a> * ('f -> 'f -> 'f)'    is not compatible with type    ''b * ('c -> 'e)'    .
Possible overload: 'static member Map.Map : (Set2<'T> * ('T -> 'U)) * _mthd:Map -> Set2<'U> when 'T : comparison and 'U : comparison'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    'Set2<'b> * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : (Set<'a0> * ('a0 -> 'a1)) * _mthd:Map -> Set<'a1> when 'a0 : comparison and 'a1 : comparison'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    'Set<'b> * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : (StringBuilder * (char -> char)) * _mthd:Map -> StringBuilder'. Type constraint mismatch. The type     'ZipList<'a> * ('b -> 'b -> 'b)'    is not compatible with type    'StringBuilder * (char -> char)'    .
Possible overload: 'static member Map.Map : (string * (char -> char)) * _mthd:Map -> string'. Type constraint mismatch. The type     'ZipList<'a> * ('b -> 'b -> 'b)'    is not compatible with type    'string * (char -> char)'    .
Possible overload: 'static member Map.Map : (ResizeArray<'T> * ('T -> 'U)) * _mthd:Map -> ResizeArray<'U>'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    'ResizeArray<'b> * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : (Quotations.Expr<'T> * ('T -> 'U)) * _mthd:Map -> Quotations.Expr<'U>'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    'Quotations.Expr<'b> * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : (System.Collections.Generic.Dictionary<'Key,'T> * ('T -> 'U)) * _mthd:Map -> System.Collections.Generic.Dictionary<'Key,'U> when 'Key : equality'. Type constraint mismatch. The type     'ZipList<'a> * ('e -> 'e -> 'e)'    is not compatible with type    'System.Collections.Generic.Dictionary<'b,'c> * ('c -> 'd)'    .
Possible overload: 'static member Map.Map : (Map<'Key,'T> * ('T -> 'U)) * _mthd:Map -> Map<'Key,'U> when 'Key : comparison'. Type constraint mismatch. The type     'ZipList<'a> * ('e -> 'e -> 'e)'    is not compatible with type    'Map<'b,'c> * ('c -> 'd)'    .
Possible overload: 'static member Map.Map : (System.Collections.Generic.KeyValuePair<'a0,'T> * ('T -> 'U)) * _mthd:Map -> System.Collections.Generic.KeyValuePair<'a0,'U>'. Type constraint mismatch. The type     'ZipList<'a> * ('e -> 'e -> 'e)'    is not compatible with type    'System.Collections.Generic.KeyValuePair<'b,'c> * ('c -> 'd)'    .
Possible overload: 'static member Map.Map : (Choice<'T,'E> * ('T -> 'U)) * _mthd:Map -> Choice<'U,'E>'. Type constraint mismatch. The type     'ZipList<'a> * ('e -> 'e -> 'e)'    is not compatible with type    'Choice<'b,'c> * ('b -> 'd)'    .
Possible overload: 'static member Map.Map : (Result<'T,'E> * ('T -> 'U)) * _mthd:Map -> Result<'U,'E>'. Type constraint mismatch. The type     'ZipList<'a> * ('e -> 'e -> 'e)'    is not compatible with type    'Result<'b,'c> * ('b -> 'd)'    .
Possible overload: 'static member Map.Map : (Async<'T> * ('T -> 'U)) * _mthd:Map -> Async<'U>'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    'Async<'b> * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : ('T [,,,] * ('T -> 'U)) * _mthd:Map -> 'U [,,,]'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    ''b [,,,] * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : ('T [,,] * ('T -> 'U)) * _mthd:Map -> 'U [,,]'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    ''b [,,] * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : ('T [,] * ('T -> 'U)) * _mthd:Map -> 'U [,]'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    ''b [,] * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : ('T [] * ('T -> 'U)) * _mthd:Map -> 'U []'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    ''b [] * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : (('Monoid * 'T) * ('T -> 'U)) * _mthd:Map -> 'Monoid * 'U'. Type constraint mismatch. The type     'ZipList<'a> * ('e -> 'e -> 'e)'    is not compatible with type    '('b * 'c) * ('c -> 'd)'    .
Possible overload: 'static member Map.Map : (System.Func<'R,'T> * ('T -> 'U)) * _mthd:Map -> System.Func<'R,'U>'. Type constraint mismatch. The type     'ZipList<'a> * ('e -> 'e -> 'e)'    is not compatible with type    'System.Func<'b,'c> * ('c -> 'd)'    .
Possible overload: 'static member Map.Map : (('R -> 'T) * ('T -> 'U)) * _mthd:Map -> ('R -> 'U)'. Type constraint mismatch. The type     'ZipList<'a> * ('e -> 'e -> 'e)'    is not compatible with type    '('b -> 'c) * ('c -> 'd)'    .
Possible overload: 'static member Map.Map : ('T list * ('T -> 'U)) * _mthd:Map -> 'U list'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    ''b list * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : ('T option * ('T -> 'U)) * _mthd:Map -> 'U option'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    ''b option * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : (System.Threading.Tasks.Task<'T> * ('T -> 'U)) * _mthd:Map -> System.Threading.Tasks.Task<'U>'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    'System.Threading.Tasks.Task<'b> * ('b -> 'c)'    .
Possible overload: 'static member Map.Map : (System.Lazy<'T> * ('T -> 'U)) * _mthd:Map -> System.Lazy<'U>'. Type constraint mismatch. The type     'ZipList<'a> * ('d -> 'd -> 'd)'    is not compatible with type    'System.Lazy<'b> * ('b -> 'c)'    .
Possible overload: 'static member ZipList.Map : ZipList<'a> * f:('a -> 'b) -> ZipList<'b>'. Type constraint mismatch. The type     'ZipList<'a> * ('c -> 'c -> 'c)'    is not compatible with type    'ZipList<'b>'    .

[gusty]

It seems to me that it's trying to match the first trait-call for Map.Invoke which has tupled arguments (to avoid subsumption) with that Map member defined on ZipList.

the one that should resolve is this one:

Possible overload: 'static member Map.Map : ( ^Functor<'T> * ('T -> 'U)) * _mthd:Internals.Default1 -> 'Functor<'U> when  ^Functor<'T> : (static member Map :  ^Functor<'T> * ('T -> 'U) -> 'Functor<'U>)'. Type constraint mismatch. The type 
    'ZipList<'a> * ('f -> 'f -> 'f)'    
is not compatible with type
    ''b * ('c -> 'd)'    �.

which contains another trait-call to the members defined on external classes, without tupling. Now, why 'ZipList<'a> * ('f -> 'f -> 'f)' doesn't match with ''b * ('c -> 'd)' have no clue.

Maybe this is a good case to test with @smoothdeveloper improvement on these error reports.

[dsyme]

Yes. It's a bit confusing that some Map instances take the extra arguments and some don't

[gusty]

It is.

Basically, all the overloads on the "Witnesses" type use dummy / tupled parameters while the ones defines in custom types have a clean signature.

[dsyme]

I've verified that the latest commit of RFC FS-1043 (https://github.com/dotnet/fsharp/pull/6805) builds and compiles the latest master of FSharpPlus, with the exception of these two lines:

type ParallelArray<'t> with
     static member inline (+) (x: parray<'m>, y: parray<'m>) = lift2 plus x y : parray<'m>

type ZipList<'s> with
    static member inline (+) (x: ZipList<'a>, y: ZipList<'a>) = lift2 plus x y : ZipList<'a>

My conclusion from previous threads on this was that this can be considered an appropriate adjustment to the specification and a code change would be needed here. It is the only place where we've found a code change may be needed

[gusty]

Could you possibly try it with https://github.com/fsprojects/FSharpPlus/pull/302 ?

[dsyme]

Could you possibly try it with #302 ?

It's ok, I'm sure we can work it out once the feature lands. The case is under test both positively and negateively now in the RFC implementation.

[dsyme]

To test this I put fsharp and FSharpPlus in parallel directories, built fsharp and added this to Directory.Build.props in FSharpPlus

  <PropertyGroup>
     <DisableAutoSetFscCompilerPath>true</DisableAutoSetFscCompilerPath>
    <FscToolPath>c:\GitHub\dsyme\fsharp2\artifacts\bin\fsc\Release\net472</FscToolPath>
    <FscToolExe>fsc.exe</FscToolExe>
    <OtherFlags>/langversion:preview $(OtherFlags)</OtherFlags>
  </PropertyGroup>

[gusty]

@dsyme

Testing with your feature/ext branch of the compiler I see lot of failures in the test projects.

Here's a relative small repro:

type Default3 = class  end
type Default2 = class inherit Default3 end
type Default1 = class inherit Default2 end

type Bind =
    static member (>>=) (source, f: 'T -> _) = Option.bind   f source
    static member (>>=) (source, f: 'T -> _) = List.collect  f source

    static member inline Invoke (source: '``Monad<'T>``) (binder: 'T -> '``Monad<'U>``) : '``Monad<'U>`` =
        let inline call (_mthd: 'M, input: 'I, _output: 'R, f) = ((^M or ^I or ^R) : (static member (>>=) : _*_ -> _) input, f)
        call (Unchecked.defaultof<Bind>, source, Unchecked.defaultof<'``Monad<'U>``>, binder)


type Return =

    static member inline Invoke (x: 'T) : '``Applicative<'T>`` =
        let inline call (mthd: ^M, output: ^R) = ((^M or ^R) : (static member Return : _*_ -> _) output, mthd)
        call (Unchecked.defaultof<Return>, Unchecked.defaultof<'``Applicative<'T>``>) x
    
    static member Return (_: option<'a> , _: Return  ) = fun x -> Some x
    static member Return (_: list<'a>       , _: Return  ) = fun x -> [ x ]


type Delay =
    inherit Default1
    
    static member inline Delay (_mthd: Default3, x: unit-> ^``Monad<'T>`` , _: obj) = Bind.Invoke (Return.Invoke ()) x : ^``Monad<'T>``
    static member        Delay (_mthd: Default2, x: unit-> _  , _     ) = Seq.delay x
    static member        Delay (_mthd: Delay   , x: unit-> _    , _     ) = async.Delay x

    static member inline Invoke (source: unit -> 'R) : 'R =
        let inline call (mthd: ^M, input: unit -> ^I) = ((^M or ^I) : (static member Delay : _*_*_ -> _) mthd, input, Unchecked.defaultof<Delay>)
        call (Unchecked.defaultof<Delay>, source)


let lst2: _ list = Delay.Invoke (fun () -> Return.Invoke 6) // fails

I updated to the latest commit of the feature/ext branch but still fails.