ITK icon indicating copy to clipboard operation
ITK copied to clipboard

`itkStatisticsUniqueLabelMapFilterTest1` sometimes fails in CI

Open tbirdso opened this issue 3 years ago • 3 comments

Description

itkStatisticsUniqueLabelMapFilterTest1 recently started failing in CI. Observed failures on both Windows and macOS.

Steps to Reproduce

Run itkStatisticsUniqueLabelMapFilterTest1 and compare output to baseline

Expected behavior

Test passes

Actual behavior

Output image differs from baseline

2022-05-25T18:35:15.2139324Z 1883: Trying writer->Update()
2022-05-25T18:35:15.2140727Z 1883: <DartMeasurement name="ImageError cthead1Label-label-statistics-unique-labelmap-baseline1.png" type="numeric/double">15</DartMeasurement>
2022-05-25T18:35:15.2142370Z 1883: <DartMeasurement name="ImageError" type="numeric/double">15</DartMeasurement>
2022-05-25T18:35:15.2143798Z 1883: <DartMeasurement name="ImageError Minimum" type="numeric/double">82</DartMeasurement>
2022-05-25T18:35:15.2145256Z 1883: <DartMeasurement name="ImageError Maximum" type="numeric/double">82</DartMeasurement>
2022-05-25T18:35:15.2146854Z 1883: <DartMeasurement name="ImageError Mean" type="numeric/double">82</DartMeasurement>
2022-05-25T18:35:15.2175541Z 1883: <DartMeasurementFile name="DifferenceImage" type="image/png">D:/a/1/s-build/Testing/Temporary/itkStatisticsUniqueLabelMapFilterTest1.png.diff.png</DartMeasurementFile>
2022-05-25T18:35:15.2177873Z 1883: <DartMeasurementFile name="BaselineImage" type="image/png">D:/a/1/s-build/Testing/Temporary/itkStatisticsUniqueLabelMapFilterTest1.png.base.png</DartMeasurementFile>
2022-05-25T18:35:15.2180062Z 1883: <DartMeasurementFile name="TestImage" type="image/png">D:/a/1/s-build/Testing/Temporary/itkStatisticsUniqueLabelMapFilterTest1.png.test.png</DartMeasurementFile>
2022-05-25T18:35:15.2182290Z 1883: <DartMeasurement name="BaselineImageName" type="text/string">cthead1Label-label-statistics-unique-labelmap-baseline1.png</DartMeasurement>
2022-05-25T18:35:15.2278622Z 1884: Trying writer->Update()
2022-05-25T18:35:15.2290186Z 1884: libpng warning: sCAL: invalid unit
2022-05-25T18:35:15.2301284Z 1884: libpng warning: sCAL: invalid unit
2022-05-25T18:35:15.2306662Z 1883: <DartMeasurement name="ImageError cthead1Label-label-statistics-unique-labelmap-dilate-baseline1.png" type="numeric/double">0</DartMeasurement>
2022-05-25T18:35:15.2339238Z 1883: <DartMeasurement name="BaselineImageName" type="text/string">cthead1Label-label-statistics-unique-labelmap-dilate-baseline1.png</DartMeasurement>
2022-05-25T18:35:15.2653717Z 1884/2779 Test #1883: itkStatisticsUniqueLabelMapFilterTest1 .................................................................***Failed    0.24 sec

Reproducibility

Versions

master

Environment

macOS failure in CDash

Windows failure in CDash

Additional Information

tbirdso avatar May 25 '22 19:05 tbirdso

@dzenanz Any thoughts on this one? I do not see any recent changes in itk::StatisticsUniqueLabelMap files.

tbirdso avatar May 25 '22 19:05 tbirdso

This test was sporadically failing in nightly dashboard. As the CI is green now, I guess this was a sporadic failure in the CI.

dzenanz avatar May 26 '22 07:05 dzenanz

Strange, but good that it is passing now. I will leave this open as a data point in the event that it happens again.

tbirdso avatar May 26 '22 12:05 tbirdso