KdTree icon indicating copy to clipboard operation
KdTree copied to clipboard

Serialization when saving a large tree. Workaround is to use FsPickler

Open fwaris opened this issue 3 years ago • 0 comments

Saving a

    let t1 =  new KdTree<float, Alrm>(1, new DoubleMath());
    t1.SaveToFile(@"D:\alarms\alarmIndex.bin")

Gives the following error with about 7M entries: System.Runtime.Serialization.SerializationException: The internal array cannot expand to greater than Int32.MaxValue elements. at System.Runtime.Serialization.ObjectIDGenerator.Rehash() at System.Runtime.Serialization.ObjectIDGenerator.GetId(Object obj, Boolean& firstTime)

However this worked for both saving and loading:

    let ser = MBrace.FsPickler.BinarySerializer()
    ser.Serialize(IO.File.Create @"D:\alarms\alarmIndex.bin", t1)
    let t2:KdTree<float, Alrm> = ser.Deserialize(IO.File.OpenRead @"D:\alarms\alarmIndex.bin")

Maybe you can switch to FsPickler internally for serialization

fwaris avatar Apr 01 '21 22:04 fwaris