Inject random data in a setup() method too

[moifort]

Hey everybody!

First awesome project! congrats for the work done!!!

I have a feature to ask but before I want to know if it's logical or not. I want to add random data in the setUp() method. I explain why, I practice test and often I need to add in the before class a lot of data to load my repository by a set of data and I want to know if it's possible to make something like that:

@RunWith(JUnitQuickcheck.class)
public class PublicationServiceTest {

    private PublicationService publicationService;

    @Before // <-- specific quickCheck annotation
    public void setUp(List<PublicationImpl> publications) { // Random set of data
        PublicationRepositoryInMemory publicationRepositoryInMemory = new PublicationRepositoryInMemory(publications);
        publicationService = new PublicationServiceImpl(new PublicationRepositoryInMemory(publications));
    }

    // Tests
}

[pholser]

@moifort Thanks for this -- sorry for the late reply.

I think this could be a worthwhile enhancement.

When the JUnitQuickcheck runner executes a test class, it runs both @Property methods and regular @Test methods. Should @Before methods injected with generated data run before regular @Test methods also, or @Property methods only?

How many values should be generated?

Should we allow multiple junit-quickcheck @Before methods? Should they run before or after all of the regular JUnit @Before methods?

If a property fails, the assertion message contains the random seeds used to produce values for the property's parameters. What should junit-quickcheck do about reporting the seeds used to produce values for junit-quickcheck @Before methods?

[pholser]

@moifort What if, instead of injecting random data into @Before methods, we provided access to an object of type Generators, just like generator classes have? Maybe you could then say something like this in your @Before method:

@RunWith(JUnitQuickcheck.class)
public class PublicationServiceTest {
    @ClassRule public final GeneratorRule gen = new GeneratorRule();

    private PublicationService publicationService;

    @Before public void setUp() {
        publicationService =
            new PublicationServiceImpl(
                new PublicationRepositoryInMemory(
                    gen.type(Publication.class).times(100)
                )
            );
    }
}

[pholser]

Wait, that sucks. times(100) needs generate called on it with a SourceOfRandomness and a GenerationStatus, which you'd have to provide. 😦

Let me think about this some more.

In the meantime, your property methods could have extra parameters of type List<Publication>.

[m0smith]

It might be nice to have annotations @BeforeProperty and @AfterProperty that would execute similar to @Before and @After except be able to take parameters like @Property.

They could have an "order" field that would tell what order to execute them. The default order would be 0, which is where the @Before and @After would run. The order would be ascending for @BeforeProperty and descending for @AfterProperty.

For example,

    @Before public void before(){}

    @BeforeProperty(order=-1) public void preBefore(String name){}

    @BeforeProperty(order=1) public void postBefore(int value){}

    @After public void after(){}

    @AfterProperty(order=-1) public void preAfter(String name){}

   @AfterProperty(order=1) public void postAfter(int value){}

would execute in preBefore, before, postBefore [run test] postAfter, after, preAfter order.

This in conjunction to a ReferencedGenerator (I will post to the forum) would add a lot of value.

[pholser]

@moifort @m0smith I'm thinking about a special kind of Rule that can be primed with random values. Rules can be chained in whatever order you like using RuleChain. I'm going to see where this takes me, and get back to you.

[moifort]

I am so sorry, I just see your message, (after half a year 😩)!

It could be a good idea! 😃, thank you for the answers!

By the way have a nice end of year!

[pholser]

@moifort @m0smith @spodkowinski At your convenience, please have a look at branch https://github.com/pholser/junit-quickcheck/tree/issues/113/property-rule. This is a proof of concept regarding a special Rule that can be fed random data generated by junit-quickcheck.

You declare rules on your test class:

    @RunWith(JUnitQuickcheck.class)
    public static class WithPropertyRule {
        private Foo f;

        @Rule public final PropertyRule p1 = new PropertyRule() {
            public void prime(Foo f) {
                System.out.printf("field prime %s\n", f);
                WithPropertyRule.this.f = f;
            }
        };

        @Rule public final PropertyRule p2() {
            return new PropertyRule() {
                public void prime(Foo f) {
                    System.out.printf("method prime %s\n", f);
                }
            };
        }

        @Property public void holds(Box<Foo> b) {
            System.out.printf("%s %s\n", f, b);
        }
    }

junit-quickcheck finds all the rules of type PropertyRule, initializes them as necessary, then looks for a method named prime on the rules, and generates values for prime's parameters just as it does for the property methods. Annotations on the parameters are honored as usual.

You are free to do whatever you wish in prime. If you want to "dispose" of them (like if they're sockets or files), override after() and do what you need to. If you want property methods to be able to access these values, stash them where the property methods can reach them.

There are some complicating factors.

1. If a property fails, the assertion message contains the random seeds used to produce values for the property's parameters. As things stand now, the seeds used to produce values for the PropertyRules will not be so reported. This would complicate being able to reproduce failures for debugging purposes.

2. You cannot use a PropertyRule as a @ClassRule, only as a @Rule. I doubt this would hinder most users. I don't have any machinery to warn the programmer about this condition; currently running such a test fails pretty spectacularly without much help from the stack trace/error message.

3. If a property fails on a certain set of values, junit-quickcheck can attempt a series of "shrinks" to try to find a small counterexample. As things stand right now, on a shrink attempt the PropertyRules would run again, with a fresh set of random values handed to them, without them being "shrunken". I'm not sure that this is very helpful. We could probably make junit-quickcheck get around this, but it seems pretty hacky.

With all this said, I'm not sure this is much better than having extra parameters on a property method.

Let me know what you think. Thanks!

[m0smith]

@pholser I really like the idea of using the @Rule annotation to take care of the before/after semantics.

A common use case I have is to create random files on disk and the TemporaryFolder Role does that nicely except the names of the files need a certain extension (html, css, etc). To your point, I could do something like:

@RunWith(JUnitQuickcheck.class)
public class TestBar {
    @Rule  public TemporaryFolder tf = new TemporaryFolder();

   @Property testSomething(@From (FileNameGenerator.class) List<String> names) {
        for( String name: names) { tf.newFile(name);}

       //  Do a cool test
   }
}

I like that a lot except the rule initialization gets dragged into the property and will need to be repeated for each property. I do not think this would even require the PropertyRule object to work. Just adding Rule support.

What about making PropertyRule an interface or annotation that can be applied to any @Rule? Instead of a hardwired "prime" method add a @Prime annotation.

// A marker interface
public interface PropertyRule {}

public class TemporaryFolderProperty extends TemporaryFolder implements PropertyRule {

    @Prime public void prime(@From (FileNameGenerator.class) List<String> names) {
        for( String name: names) { this.newFile(name);}
      }   
}


@RunWith(JUnitQuickcheck.class)
public class TestBar {
    @Rule  public TemporaryFolderProperty folder= new TemporaryFolderProperty();

   @Property testSomething() {
           //  Do a cool test
   }
}

This would allow any Rule to play .

My first example is how I would use the Rule annotation and would get around the issues you noted as all the generation is still part of the @Property. Just implementing straight Rules and documenting how to use them in conjunction with the @Property would be a great addition.

Later, a PropertyRule interface or annotation could be added once the issues you mentioned are addressed.

Thanks again.