machinelearning
machinelearning copied to clipboard
BufferBuilder.MakeDense: IndexOutOfRangeException
System Information:
- OS & Version: Windows 11
- ML.NET Version: 1.6.0 (custom build with autoML search space edited)
- .NET Version: .NET 5.0
Describe the bug I am running AutoML. The below is from logs, probably about 1 out of 50 runs on same dataset.
To Reproduce Unknown. It might be related to specific model parameters, but they were not logged (the search space is customized). Unfortunately, not even model is saved, since I am only exploring the weights. Model was trained succesfully.
Screenshots, Code, Sample Projects Although, global feature index works almost always, I have this exception in logs.
System.IndexOutOfRangeException: Index was outside the bounds of the array.
at Microsoft.ML.Data.BufferBuilder`1.MakeDense() in Q:\git-kwork-microsoft-ml\Microsoft.ML\src\Microsoft.ML.Data\Data\BufferBuilder.cs:line 322
at Microsoft.ML.Data.BufferBuilder`1.GetResult(VBuffer`1& buffer) in Q:\git-kwork-microsoft-ml\Microsoft.ML\src\Microsoft.ML.Data\Data\BufferBuilder.cs:line 462
at Microsoft.ML.Trainers.FastTree.TreeEnsembleModelParameters.GetFeatureWeights(VBuffer`1& weights) in Q:\git-kwork-microsoft-ml\Microsoft.ML\src\Microsoft.ML.FastTree\FastTree.cs:line 3242
at Kwork.AI.AutoML.Experiment.SaveGlobalFeatureIndexFastTree(MLContext mlContext, ITransformer model, String filename, IDataView testDataView, BinaryClassificationTrainer trainer) in Q:\git-kwork-microsoft-ml\Microsoft.ML\src-AutoML-Runner\Kwork.AI.AutoML.Runner\Experiment.cs:line 668
Link to BufferBuilder.MakeDense: https://github.com/dotnet/machinelearning/blob/52ddbcd0a57b440e60e43e6cb39b49fe55bdfe55/src/Microsoft.ML.Data/Data/BufferBuilder.cs#L316
Additional context It is not blocking my work, only for FYI
Thanks for the heads up! I'm glad its not blocking. When I get a chance I see if I can get a repro, but if you can think of anything else that may help let me know.