Memory usage

[alex-thc]

Thank you for a great project! What's the expected memory usage for mast? We've been doing some simple tests and it looks like there's some non-trivial overhead in it related to keys and values:

func test2() *mast.Mast {
	ctx := context.Background()
	v1 := mast.NewInMemory()

	// Insert 1 million elements with 4 byte keys and values
	var i int32
	for i = 0; i < 1*1000*1000; i++ {
		v1.Insert(ctx, i, i)
	}

	return &v1
}

This code consumes 136MB on my machine, while the expectation is to see something closer to ~8MB (1 Mil x 8 bytes)

Oddly enough, if I replace both int32 with 4-byte arrays, the memory usage goes 3x higher to ~460MB:

func test1() *mast.Mast {
	ctx := context.Background()
	v1 := mast.NewInMemory()

	// Insert 1 million elements with 4 byte keys and values
	var i int64
	for i = 0; i < 1*1000*1000; i++ {
		//create a 4 byte array
		var val [4]byte
		val[0] = byte(i)
		val[1] = byte(i >> 8)
		val[2] = byte(i >> 16)
		val[3] = byte(i >> 24)

		var key [4]byte
		key[0] = byte(i)
		key[1] = byte(i >> 8)
		key[2] = byte(i >> 16)
		key[3] = byte(i >> 24)

		v1.Insert(ctx, key, val)
	}

	return &v1
}

[alex-thc]

To make this more odd, if I store int64 keys and byte[4] values, my memory usage drops to 72MB:

func test3() *mast.Mast {
	ctx := context.Background()
	v1 := mast.NewInMemory()

	// Insert 1 million elements with 4 byte keys and values
	var i int64
	for i = 0; i < 1*1000*1000; i++ {
		//create a 4 byte array
		var val [4]byte
		val[0] = byte(i)
		val[1] = byte(i >> 8)
		val[2] = byte(i >> 16)
		val[3] = byte(i >> 24)

		v1.Insert(ctx, i, val)
	}

	return &v1
}