Unable to aggregate multiple benchmarks of different problems as fileProblemProvider not set

[Magnusrk]

I am using the benchmark aggregator and running multiple different problems from xml files. However when i try to aggregate them together i get this error.

Caused by: java.lang.NullPointerException: Cannot invoke "java.io.File.toString()" because "this.problemFile" is null at ai.timefold.solver.benchmark.impl.loader.FileProblemProvider.toString(FileProblemProvider.java:74)

In the aggregator java app i can see that all results have been put into the last problems files (see last screenshot). This is not the case in my benchmark report folder here everything looks correct. I then looked in the plannerBenchMarkResult.xml file and found that the fileProblemProvider had not been set for any problem.

  <unifiedProblemBenchmarkResult>
    <name>2units2Areas-differentCapabilities</name>
    <fileProblemProvider/>
    <writeOutputSolutionEnabled>true</writeOutputSolutionEnabled>

I went through the code in debug mode and found that in the BenchmarkResultIO class when it writes the problem to the file does not include the fileProblemProvider even though it exist inside the unifiedProblemBenchmarkResultList.

To Reproduce

    public static void main(String[] args)  {
        // Create the PlannerBenchmarkFactory from the XML configuration
        PlannerBenchmarkFactory benchmarkFactory =
                PlannerBenchmarkFactory.createFromXmlResource("solverBenchmarkConfig.xml");

        // Let Timefold load problems from XML files via SolutionFileIO
        PlannerBenchmark benchmark = benchmarkFactory.buildPlannerBenchmark();

        // Run the benchmark and generate the report (including output solution files)
        benchmark.benchmarkAndShowReportInBrowser();
        BenchmarkAggregatorFrame.createAndDisplayFromXmlResource("solverBenchmarkConfig.xml");
    }

Here is part of my solverBenchMarkConfig.xml

        <problemBenchmarks>
            <solutionFileIOClass>com.mcm.planning.debugging.MCMPlanFileIO</solutionFileIOClass>
            <writeOutputSolutionEnabled>true</writeOutputSolutionEnabled>

            <inputSolutionFile>src/main/resources/xml/MineSweeping/2units2Areas-sameCapabilities.xml</inputSolutionFile>
            <inputSolutionFile>src/main/resources/xml/MineSweeping/2units2Areas-differentCapabilities.xml</inputSolutionFile>


            <problemStatisticType>BEST_SCORE</problemStatisticType>
            <problemStatisticType>STEP_SCORE</problemStatisticType>
            <problemStatisticType>MEMORY_USE</problemStatisticType>
        </problemBenchmarks>

Environment

Timefold Solver Version or Git ref: 1.19.0

Output of java -version: openjdk version "24" 2025-03-18 OpenJDK Runtime Environment Zulu24.28+83-CA (build 24+36) OpenJDK 64-Bit Server VM Zulu24.28+83-CA (build 24+36, mixed mode, sharing)

Additional information

Here are some screenshots of the debug mode.

[triceo]

Thanks for reporting, @Magnusrk! We'll take a look when time permits.

[Magnusrk]

problem can be fixed by adding @XmlAccessorType(XmlAccessType.FIELD) to the FileProblemProvider class

If that is the case, then a simple addition of a getter/setter pair will do the trick as well.