machinelearning icon indicating copy to clipboard operation
machinelearning copied to clipboard

DataFrameColumn: Unable to Filter by Null via ElementWiseXYZ Methods (System.NotSupportedException)

Open JD-Robbs opened this issue 2 years ago • 0 comments

System Information (please complete the following information):

  • OS & Version: Windows 10
  • ML.NET Version: Latest
  • .NET Version: .NET 6.0

Describe the bug When passing null into ElementWiseXYZ methods, a System.NotSupportedException is thrown: 'Specified method is not supported.'

To Reproduce Steps to reproduce the behavior:

        var col = new DoubleDataFrameColumn("col", new double?[] {1.23, null, 2, 3 });
        var dfTest = new DataFrame(col);

        var filtered2Df = dfTest.Filter(dfTest["col"].ElementwiseNotEquals(2));
        var filteredNullDf = dfTest.Filter(dfTest["col"].ElementwiseNotEquals(null));

Expected behavior I expected the resulting DataFrame stored in filteredNullDf to not include the value null - much like filered2Df does not include the value 2.

The only way of making it work is something like this:

         dfTest.Filter(new PrimitiveDataFrameColumn<bool>("filter", dfTest["col"].Cast<double?>().Select(v => v != null)));

JD-Robbs avatar Mar 03 '22 10:03 JD-Robbs