jqwik icon indicating copy to clipboard operation
jqwik copied to clipboard

Allowing Sampling of Arbitraries with Injected Seed

Open gypsydave5 opened this issue 1 year ago • 1 comments

Testing Problem

I like using random data in my tests as often as possible - so if a test fixture doesn't need to be a particular value for the test, I set it to some random instance.

jqwik Arbitraries are great for this! Just call Arbitrary.sample() and away we go, they compose beautifully. Really enjoy it.

However - because I can't control the randomness that sample is using, it means that it's harder to make the tests repeatable (I'd have to output some representation of the sampled object).

Suggested Solution

I was originally thinking that I'd want to inject the seed directly to sample, but taking a look at the JqwikSession api, and the two methods in the TODO file setRandomSessionSeed(), getRandomSessionSeed(), can I infer that this is the intended direction of travel - so set up my session, use it in the test(s), output the seed on failure, and inject it when I want to repeat?

Discussion

I'm guessing the solution is for me to shut up and wait / offer to help out? 😄

gypsydave5 avatar Jun 30 '24 20:06 gypsydave5

You can do it with a bit more verboseness:

Arbitrary<String> strings = Arbitraries.strings().withCharRange('a', 'z').ofMinLength(1);
Random random = new Random(42);
JqwikSession.run(() -> {
	strings.generator(1000)
		   .stream(random).limit(10)
		   .map(Shrinkable::value)
		   .forEach(System.out::println);
});

That said, I'll set the session methods for JqwikSession on the TODO list for 1.9.1 since there's nothing urgent there and all big features will have to wait for Jqwik 2.

jlink avatar Jul 01 '24 06:07 jlink

I suggest two new overloaded methods:

JqwikSession.start(String randomSeed) and JqwikSession.run(String randomSeed, Runnable sessionCode).

@gypsydave5 Would that fulfill your needs?

jlink avatar Jul 10 '24 13:07 jlink

I added three new methods to JqwikSession:

/**
 * Returns the {@linkplain Random} instance associated with the current session.
 * @return a Random instance if a session is active, otherwise an empty Optional
 */
@API(status = EXPERIMENTAL, since = "1.9.1")
public static Optional<Random> getRandom() {...}

/**
 * Starts a new session with a given random seed.
 * Currently seeds must be strings that can be parsed by {@linkplain Long#parseLong(String)}.
 */
@API(status = EXPERIMENTAL, since = "1.9.1")
public static void start(String randomSeed) {...}

/**
 * Runs a given {@linkplain Runnable} in a new session with a given random seed.
 * Currently seeds must be strings that can be parsed by {@linkplain Long#parseLong(String)}.
 */
@API(status = EXPERIMENTAL, since = "1.9.1")
public static void run(String randomSeed, Runnable runnable) {...}

They are available in 1.9.1-SNAPSHOT.

@gypsydave5 Maybe you want to try it out if you find the time.

jlink avatar Jul 12 '24 08:07 jlink

Feel free to reopen if any problems show up.

jlink avatar Jul 18 '24 09:07 jlink

Those features have just been released in version 1.9.1: https://jqwik.net/release-notes.html#191

jlink avatar Sep 30 '24 12:09 jlink

Sorry - terribly rude of me! Thank you for all of the above, very much suits my use 😄 - it's appreciated!

gypsydave5 avatar Nov 08 '24 12:11 gypsydave5