JUnit 4.12 converts null string parameters to "null"s

[MGhareeb32]

When running the following test with junitLibVersion = "4.12":

import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;

import java.util.Arrays;

import static org.junit.Assert.assertEquals;

@RunWith(Parameterized.class)
public class TestNull {

    private final String mInput;

    @Parameters
    public static Iterable<Object[]> data() {
        return Arrays.asList(new Object[][] { {null} });
    }

    public TestNull(String input) {
        mInput = input;
    }
    @Test
    public void test() {
        assertEquals(null, mInput);
    }
}

I get the following assertion failure:

java.lang.AssertionError: expected: null<null> but was: java.lang.String<null>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at com.myapp.android.regression.TestNull.test(TestNull.java:27)
at java.lang.reflect.Method.invoke(Native Method)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:59)
at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:262)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1879)

[stefanbirkner]

I created a project stefanbirkner/junit4-issue-1488 for verifying the issue.

I cannot reproduce the failure with Java 8. @MGhareeb32 Can you please verify that the example project is failing for you, too.

[kcooney]

Thanks, @stefanbirkner ; I was going to do the same thing :-)

I noticed Android classes on the call stack. @MGhareeb32 does this happen for projects that don't depend on Android?

[MGhareeb32]

Thanks @stefanbirkner and @kcooney. Pulling the project and trying to reproduce...

[MGhareeb32]

I'm afraid I'm not able to reproduce outside of the Android setup I have. I tried to change @stefanbirkner's repo to use the same dependencies my Android repo has, but no luck.

[stefanbirkner]

@MGhareeb32 Can you change my repo so that it has an Android setup?

[MGhareeb32]

Will get back to you later today :)

[kcooney]

If you are able to launch your tests in a debugger, I suggest doing that.

[npmccallum]

I just ran into this today. It was very painful.

[kcooney]

@npmccallum we only closed this because no one could reproduce it with vanilla JUnit 4.12. Could you provide a reproducible test case?

[dabrosch]

I also just ran into this today, and it specifically occurs when I run my JUnit Android Espresso test on an Emulator running API 23, but not with 26 or greater.

Building on Java 8, with Android SDK 28.

[kcooney]

@dabrosch my guess is that this is Android-specific, which is why the JUnit team hasn't been able to reproduce it. Could you use a debugger to try to figure out why you are getting this failure message?

[dabrosch]

I can confirm that the result of Arrays.asList(new Object[][] { {null} } is the exact same when executing on Android API 23 or 26, therefore it must be in the Parameter assignment to a string that is coercing it into a String "null."