Incompatibility between Specifications, NullPointerException Handling, and Precondition Enforcement

[DaisyPo]

Environment

The version of Randoop: randoop-all-4.3.3.jar java -version: Java 8 operating system version: Ubuntu 20.04

Description

I've encountered two relevant issues while using Randoop for Java test case generation, specifically when using specifications and handling NullPointerException.

Issue 1: When I use specifications, command such as --npe-on-null-input=INVALID doesn't work.

Despite setting the following options to mark NullPointerException as invalid:

--npe-on-null-input=INVALID
--forbid-null=true
--checked-exception=INVALID

Randoop still generates an error-revealing test that passes null as an argument, which eventually throws a NullPointerException.

The complete commands (The code of the target project is attached in the compressed file. spec.json and classlist.txt are included.): java -classpath path/to/randoop-all-4.3.3.jar:target/classes randoop.main.Main gentests --classlist=classlist.txt --npe-on-null-input=INVALID --checked-exception=INVALID --forbid-null=true --specifications=spec.json --junit-output-dir=randoop_test --no-regression-tests=true

The project is modified from commons-math3-3.6.1-src. I use mvn compile to compile it.

What I expect: Since I inject one fault at org.apache.commons.math3.analysis.polynomials.PolynomialFunctionLagrangeForm.verifyInterpolationArray: Original code:

if (x.length < 2){
    throw new NumberIsTooSmallException(LocalizedFormats.WRONG_NUMBER_OF_POINTS, 2, x.length, true);
}

Modified code:

if (x.length == 1) {
    throw new NumberIsTooSmallException(LocalizedFormats.WRONG_NUMBER_OF_POINTS, 2, x.length, true);
}

I expect one error-revealing test to violate this condition:

     {
        "exception": "org.apache.commons.math3.exception.NumberIsTooSmallException",
        "description": "@throws org.apache.commons.math3.exception.NumberIsTooSmallException if the number of points is less than 2.",
        "guard": {
          "condition": "x!=null && x.length<2",
          "description": "if the number of points is less than 2."
        }
      }

A test like the one below would meet my expectations:

@Test
public void test01() throws Throwable {

    try {
        org.apache.commons.math3.analysis.interpolation.DividedDifferenceInterpolator dividedDifferenceInterpolator0 = new org.apache.commons.math3.analysis.interpolation.DividedDifferenceInterpolator();
        double[] doubleArray2 = new double[] {};
        double[] doubleArray3 = new double[] {};

        org.apache.commons.math3.analysis.polynomials.PolynomialFunctionNewtonForm polynomialFunctionNewtonForm4 = dividedDifferenceInterpolator0.interpolate(doubleArray2, doubleArray3);

        org.junit.Assert.fail("Expected org.apache.commons.math3.exception.NumberIsTooSmallException to be thrown");
    } catch (org.apache.commons.math3.exception.NumberIsTooSmallException e) {

    }
}

However, all the error-revealing tests generated by Randoop passes null as the second argument for the method interpolate(double [], double[]) (see randoop_test).

Issue 2: Precondition Enforcement Failure

In the same specification file spec.json, I also set pre-conditions:

    "pre": [
      {
        "description": "@param double[] Interpolating points array.",
        "guard": {
          "condition": "x!=null",
          "description": "Interpolating points array."
        }
      },
      {
        "description": "@param double[] Interpolating values array.",
        "guard": {
          "condition": "y!=null",
          "description": "Interpolating values array."
        }
      }
    ]

The Randoop manual states: "If the guard of any pre-condition is violated before the call, then Randoop classifies the method call as invalid and the sequence is discarded." However, all the error-revealing tests pass null to this method. It seems that the precondition was not properly enforced, leading to a test that should have been discarded.

I would greatly appreciate any guidance or fixes for these issues. If additional information or clarification is needed, please let me know. Thank you for your time and assistance. commons-math3-3.6.1-src.zip