machinelearning icon indicating copy to clipboard operation
machinelearning copied to clipboard

BufferBuilder.MakeDense: IndexOutOfRangeException

Open torronen opened this issue 3 years ago • 1 comments

System Information:

  • OS & Version: Windows 11
  • ML.NET Version: 1.6.0 (custom build with autoML search space edited)
  • .NET Version: .NET 5.0

Describe the bug I am running AutoML. The below is from logs, probably about 1 out of 50 runs on same dataset.

To Reproduce Unknown. It might be related to specific model parameters, but they were not logged (the search space is customized). Unfortunately, not even model is saved, since I am only exploring the weights. Model was trained succesfully.

Screenshots, Code, Sample Projects Although, global feature index works almost always, I have this exception in logs.

System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at Microsoft.ML.Data.BufferBuilder`1.MakeDense() in Q:\git-kwork-microsoft-ml\Microsoft.ML\src\Microsoft.ML.Data\Data\BufferBuilder.cs:line 322
   at Microsoft.ML.Data.BufferBuilder`1.GetResult(VBuffer`1& buffer) in Q:\git-kwork-microsoft-ml\Microsoft.ML\src\Microsoft.ML.Data\Data\BufferBuilder.cs:line 462
   at Microsoft.ML.Trainers.FastTree.TreeEnsembleModelParameters.GetFeatureWeights(VBuffer`1& weights) in Q:\git-kwork-microsoft-ml\Microsoft.ML\src\Microsoft.ML.FastTree\FastTree.cs:line 3242
   at Kwork.AI.AutoML.Experiment.SaveGlobalFeatureIndexFastTree(MLContext mlContext, ITransformer model, String filename, IDataView testDataView, BinaryClassificationTrainer trainer) in Q:\git-kwork-microsoft-ml\Microsoft.ML\src-AutoML-Runner\Kwork.AI.AutoML.Runner\Experiment.cs:line 668

Link to BufferBuilder.MakeDense: https://github.com/dotnet/machinelearning/blob/52ddbcd0a57b440e60e43e6cb39b49fe55bdfe55/src/Microsoft.ML.Data/Data/BufferBuilder.cs#L316

Additional context It is not blocking my work, only for FYI

torronen avatar Jan 01 '22 17:01 torronen

Thanks for the heads up! I'm glad its not blocking. When I get a chance I see if I can get a repro, but if you can think of anything else that may help let me know.

michaelgsharp avatar Jan 10 '22 18:01 michaelgsharp