testng icon indicating copy to clipboard operation
testng copied to clipboard

Use Arrays.deepToString for rendering method parameters

Open vlsi opened this issue 3 years ago • 5 comments

TestNG Version

The latest

Expected behavior

There should be a way to understand the actual parameter for the test.

Actual behavior

FAILURE   1.0sec, test.dependent.DependentTest > simpleCyclingDependencyShouldWorkWitParallelism[3](class test.dependent.github1380.GitHub1380Sample4, [[Ljava.lang.String;@e9e1768)
    java.lang.AssertionError: 
    Expecting:
      <["testMethodB", "testMethodA", "testMethodC"]>
    to match given predicate.
    
    You can use 'matches(Predicate p, String description)' to have a better error message
    For example:
      assertThat(player).matches(p -> p.isRookie(), "is rookie");
    will give an error message looking like:
    
    Expecting:
      <player>
    to match 'is rookie' predicate
        at test.dependent.DependentTest.simpleCyclingDependencyShouldWorkWitParallelism(DependentTest.java:205)
          5.0sec, test.dependent.DependentTest > simpleCyclingDependencyShouldWorkWithoutParallelism[1](class test.dependent.github1380.GitHub1380Sample2, [Ljava.lang.String;@63c8f7f1)

Test case sample

test.dependent.DependentTest#simpleCyclingDependencyShouldWorkWitParallelism

  @Test(dataProvider = "dp1380Parallel", description = "GITHUB-1380")
  public void simpleCyclingDependencyShouldWorkWitParallelism(
      Class<?> testClass, String[] ...runMethods) { // <-- this array should render appropriately

vlsi avatar May 23 '21 12:05 vlsi

Is everything fixed by #2548?

juherr avatar May 26 '21 21:05 juherr

No, this issue is still open.

vlsi avatar May 26 '21 21:05 vlsi

The gotcha here is that deepToString might become too long (even multi-megabyte in case of long arrays in test data), so there should probably be a hard limit on the length of the resulting message, so I think the solution needs to be discussed first.

I had a similar case with JUnit where long array in parameterized test: https://github.com/junit-team/junit5/issues/2358

vlsi avatar May 26 '21 21:05 vlsi

Right. What solution do you recommend?

juherr avatar May 27 '21 10:05 juherr

I've no strong opinion here.

I thought behind the lines of 012345678910111213...truncated...sha1=12312312123, however, JUnit implemented just truncation to the first 512 characters. The sequence number is always there, so it is more-or-less enough to tell which test fails, and the checksums are not very human-readable. So the approach of [sequence] messages truncated to N chars might be ok.

vlsi avatar May 27 '21 14:05 vlsi