machinelearning
machinelearning copied to clipboard
DataFrameColumn: Unable to Filter by Null via ElementWiseXYZ Methods (System.NotSupportedException)
System Information (please complete the following information):
- OS & Version: Windows 10
- ML.NET Version: Latest
- .NET Version: .NET 6.0
Describe the bug
When passing null
into ElementWiseXYZ
methods, a System.NotSupportedException
is thrown: 'Specified method is not supported.'
To Reproduce Steps to reproduce the behavior:
var col = new DoubleDataFrameColumn("col", new double?[] {1.23, null, 2, 3 });
var dfTest = new DataFrame(col);
var filtered2Df = dfTest.Filter(dfTest["col"].ElementwiseNotEquals(2));
var filteredNullDf = dfTest.Filter(dfTest["col"].ElementwiseNotEquals(null));
Expected behavior
I expected the resulting DataFrame
stored in filteredNullDf
to not include the value null
- much like filered2Df
does not include the value 2
.
The only way of making it work is something like this:
dfTest.Filter(new PrimitiveDataFrameColumn<bool>("filter", dfTest["col"].Cast<double?>().Select(v => v != null)));