Theory / Parameterized Test Support

[farlee2121]

Have parameterized tests with manual case data been considered?

The xUnit equivalent is a theory. In NUnit it's TestCase.

I wrote a simple proof of concept for Expecto

let theory (name:string) (cases: #seq<'T>) (fTest: 'T -> 'U) =
  let dataToTest caseData =
    testCase (string caseData) <| fun () ->
      fTest caseData |> ignore
      
  testList name (cases |> Seq.map dataToTest |> List.ofSeq)
  
let theoryWithResult (name:string) (cases: #seq<('T*'U)>) (fTest: 'T -> 'U) =
  let dataToTest (caseData,expected) =
    testCase (string (caseData, expected)) <| fun () ->
      let actual = fTest caseData
      Expect.equal actual expected $"Input: {caseData} \nExpected {expected} \nActual: {actual}"
  
  testList name (cases |> Seq.map dataToTest |> List.ofSeq)


[<Tests>]
let tests =
  testList "samples" [
    testCase "universe exists (╭ರᴥ•́)" <| fun _ ->
      let subject = true
      Expect.isTrue subject "I compute, therefore I am."
    
    theory "Do I work" [1; 2; 3] <| fun (i) ->
        Expect.notEqual i 3 "No 3s"
        
    theory "Complex type" [(1,2); (2,2)] <| fun (x,y) ->
        x+y
        
    theoryWithResult "Addition" [(1,2),3; (2,2),4; (1,1),3] <| fun (x,y) ->
        x+y
  ]

[stijnmoreels]

Yeah, had this implemented myself but didn't include the 'expected' result or the name in the test so it could combine multiple manual case data tests together (theory in theory if you will).

let testParamsMany label xs tests =
  List.collect (fun x -> List.map (fun test -> test x) tests) xs
  |> testList label

let testParams label xs test =
  testParamsMany label xs [ test ]

Which could be used:

[<Tests>]
let basic_tests =
  testList "basic tests" [
    let colors =
      [ (Color.Yellow, Color.Blue);
        (Color.Blue, Color.Yellow )]
    testParams "yellow and blue always green" colors <| fun (left, right) ->
      test $"{left} + {right} = green" {
        Expect.equal (left ||| right) Color.Green "should be green" }
  ]

But I guess you could also use a regular for loop in this which would also translate into a list of tests.

[ratsclub]

I have implemented this on a pull request. There's also a small discussion from something I noticed.

[farlee2121]

Something to worth mentioning is that the theory can be async or a task. Also, there's must be a way to focus or skip it through the f and p prefix. I wonder which implementation would make more sense.

It feels weird to implement all of it:

testTheory ftestTheory ptestTheory testTheoryTask ftestTheoryTask ptestTheoryTask testTheoryAsync ftestTheoryAsync ptestTheoryAsync testTheoryWithResult ftestTheoryWithResult ptestTheoryWithResult testTheoryWithResultTask ftestTheoryWithResultTask ptestTheoryWithResultTask testTheoryWithResultAsync ftestTheoryWithResultAsync ptestTheoryWithResultAsync

First, I notice there is no testCaseTask method. So you could probably leave that group out and eliminate 6 methods.

To consolidate more, perhaps we could define an overload that takes a test method. For example

let toTheory expectoMethod name cases test =
    let caseToTest case =
      expectoMethod (string case) <| fun () ->
        test case |> ignore

    testList name (cases |> Seq.map caseToTest |> List.ofSeq)

[<Tests>]
let t = 
    testList "Examples of different methods" [
        toTheory testCase "Normal Test Case" [1,2,3] 
        <| fun (a,b, sum) ->
            Expect.equal (a+b) sum "message"

        toTheory  ftestCase "focused" [1,2,3] 
        <| fun (a,b, sum) ->
            Expect.equal (a+b) sum "message"

        toTheory  ptestCase "ignored" [1,2,3] 
        <| fun (a,b, sum) ->
            async {
                Expect.equal (a+b) sum "message"
            }
    ]

This doesn't consolidate the async versions. Even so, it'd get us down to six total methods

testTheory
toTheory
toTheoryAsync
testTheoryWithResult
toTheoryWithResult
toTheoryWithResultAsync

Alternatively, the WithResult series could be left out. It's really just the base theory method with some pre-decided tupling and message formatting. The user can achieve the same just by how they shape their input and assertions.

That leaves us with a small-ish set of methods, even fully expanded

testTheory
ftestTheory
ptestTheory
testTheoryAsync
ftestTheoryAsync
ptestTheoryAsync

[ratsclub]

I made some changes there, may you review, please?