expecto icon indicating copy to clipboard operation
expecto copied to clipboard

Q: How to reflect FsCheck related CLI option values to the custom configuration?

Open objectx opened this issue 6 years ago • 4 comments

There seem no easy ways to inject settings from CLI when using a custom FsCheck configuration.

FsCheckConfig.defaultConfig does not reflect the values from CLI options (i.e. --fscheck-max-tests etc...), So the code following uses the default maxTests (=200) even when --fscheck-max-tests 10 supplied.

let config = { FsCheckConfig.defaultConfig with arbitrary = [typeof<MyClass>] }

let test = testPropertyWithConfig config <| fun () -> …

Please tell me how to inject CLI settings when using a custom FsCheckConfig if possible.

objectx avatar Jul 24 '19 06:07 objectx

I believe they'd (the command-line-option DU cases) have to be merged. Do you feel up for adding that support? If you want to add it, you could consider writing about how you're considering architecting the new code in this issue for a bit of back-and-forth.

haf avatar Aug 20 '19 15:08 haf

Once using the custom Arbitrary, We lost parameters from the CLI option. I've struck this when testing complex tree structure (required 5min to finish with using the default maxTest settings and no way other than recompiling to change settings). This is the reason for creating this issue.

One possible design is like Fake's MSBuild module (cf. MSBuild.fs.) Using a function FsCheckConfig -> FsCheckConfig as a modifier.

objectx avatar Aug 21 '19 02:08 objectx

  • https://github.com/haf/expecto/blob/master/Expecto.FsCheck/FsCheck.fs#L89-L116

  • https://github.com/haf/expecto/blob/dc7935459c0474a9123e19721e636277725ca1dc/Expecto/Expecto.fs#L1872

  • Here: https://github.com/haf/expecto/blob/dc7935459c0474a9123e19721e636277725ca1dc/Expecto/Expecto.fs#L973-L987 you could extend with merge logic; if the config value is Some _, merge from the global config. If we make https://github.com/haf/expecto/blob/dc7935459c0474a9123e19721e636277725ca1dc/Expecto/Expecto.fs#L979-L980 optional, them being Some _ could make this snippet of code set the test's FsCheck config.

haf avatar Aug 21 '19 06:08 haf

Seems to be good enough to reflect CLI settings, and solves problems when using a custom arbitrary.

objectx avatar Aug 26 '19 05:08 objectx