Don't generate broken code

[gampleman]

Currently, if you mess up one of the formatting rules for the examples (i.e. miss a type signature for a definition), this tool will generate invalid test code, that only actually running the tests will reveal. However than one is left with the somewhat unpleasant experience of debugging generated code. Ideally this tool would actually error out in the generation phase if the code is invalid.

[stoeffel]

hmmm. do you have a specific example? It's not possible to find out everything during​ generation, because this is not a complete elm parser. But it might be possible to make some cases better.

[gampleman]

For example running on a file where parsing seems to fail (not sure why in my case) or an empty file, I get this output:

module Doc.Visualization.ScaleSpec exposing (spec)

-- This file got generated by [elm-verify-examples](https://github.com/stoeffel/elm-verify-examples).
-- Please don't modify this file by hand!

import Test
import Expect
import Visualization.Scale exposing(..)


spec : Test.Test
spec =
    Test.describe "Visualization.Scale" <|
    ]

which is not valid elm.

[stoeffel]

@gampleman I fixed an issue with modules that don't have any tests. would you mind testing with v1.0.2.?

[gampleman]

That is slightly better, although I think it would make sense for elm-verify-examples to report an error if a file doesn't have any examples, rather than leaving this to elm-test to complain about.

(I also have to figure out what's going wrong with my file which has plenty of examples, but it seems that it manages to confuse the parser somehow).

[stoeffel]

Can you share any code example? Did you add the file in your elm-verify-examples.json?

[gampleman]

This seems to not generate any examples:

module Visualization.Scale exposing (nice)

{-| Returns a new scale which extends the domain so that it lands on round values.
The second argument is the same as you would pass to ticks.

    scale : ContinuousScale
    scale = linear ( 0.5, 99 ) ( 50, 100 )
    domain (nice scale 10) --> (0, 100)

-}


nice : Int -> Int
nice a =
    a + 1

edit:

as in it generates:

module Doc.Visualization.ScaleSpec exposing (spec)

-- This file got generated by [elm-verify-examples](https://github.com/stoeffel/elm-verify-examples).
-- Please don't modify this file by hand!

import Test
import Expect
import Visualization.Scale exposing(..)


spec : Test.Test
spec =
    Test.describe "Visualization.Scale" <|
    [
    ]

[stoeffel]

The issue is that you need a newline in between the helper-function and the example.

I'm looking in how to make that work.

[stoeffel]

From the docs.

⚠️ It's important that each example is separated by a newline.

{-| This will break!

    add 1 2 --> 3
    add 1 3 --> 4
-}
{-| This works!

    add 1 2 --> 3

    add 1 3 --> 4
-}

It's a known "issue" atm. Sorry for the inconvenience.

[gampleman]

That doesn't seem to make a difference for me.

[stoeffel]

💭 hmmm... This 👇 created a test for me:

{-| Returns a new scale which extends the domain so that it lands on round values.
The second argument is the same as you would pass to ticks.

    scale : ContinuousScale
    scale = linear ( 0.5, 99 ) ( 50, 100 )

    domain (nice scale 10) --> (0, 100)

-}


nice : Int -> Int
nice a =
    a + 1

[gampleman]

hmm that's pretty strange.

This is what my tests/elm-verify-examples.json looks like:

{
  "root": "../src",
  "tests" : [
    "Visualization.List",
    "Visualization.Path",
    "Visualization.Scale"
  ]
}

[stoeffel]

@gampleman sorry totally drop the ball here. is this still an issue for you?

[gampleman]

I've encountered it again, but I suspect this might be fixed with just better error reporting, since it tends to be in large files with a lot of stuff going on. So I think if we had better parse failure messages, this would be more straightforward to diagnose.