moonblob icon indicating copy to clipboard operation
moonblob copied to clipboard

Improve table serialization efficiency by building a dictionary of the table

Open megagrump opened this issue 6 years ago • 0 comments

When serializing tables, split them into a dictionary part and an index part. The index is a sequential table that holds all unique keys and values. The dictionary resembles the original table structure, but contains only index values.

Advantages:

  • deduplication of keys and values
  • less overhead for type information
  • full support for cyclic references.

Disadvantages:

  • small performance hit
  • small tables may become slightly larger

Example:

test = {
	number1: 1
	number2: 1
	text: 'number1'
}

becomes

test = {
	index: {
		'number1'
		1
		'number2'
		'text'
	}

	dictionary: {
		1: 2
		3: 2
		4: 1
	}
}

megagrump avatar Feb 07 '19 10:02 megagrump