Add toJSON() and fromJSON() to MerkleTree

[dfstio]

It takes a long time to build MerkleTree and MerkleMap from the elements. It is 100 times faster to load the MerkleMap from the JSON file. MerkleTree.nodes is a private variable, so it is not possible to set it to the value loaded from the JSON file.

I propose to add the following functions to the MerkleTree:

  toJSON() {
    const nodes: {
      index: string;
      value: { index: string; value: string }[];
    }[] = [];
    for (const level in this.nodes) {
      const node: { index: string; value: string }[] = [];
      for (const index in this.nodes[level]) {
        node.push({ index, value: this.nodes[level][index].toJSON() });
      }
      nodes.push({ index: level, value: node });
    }
    return {
      nodes,
      height: this.height,
    };
  }

  static fromJSON(json: any) {
    const tree = new MerkleTree(json.height);
    for (const level of json.nodes) {
      for (const node of level.value) {
        tree.setNode(
          parseInt(level.index),
          BigInt(node.index),
          Field.fromJSON(node.value)
        );
      }
    }
    return tree;
  }

There is no need to add functions to MerkleMap as MerkleMap.tree is public, and it is possible to set the tree with the code

map.tree = MerkleTree.fromJSON(json);

[Pfed-prog]

I have noticed that when the tree is empty it does not take much time to serialize the tree.

In case of Merkle Tree, the json file might be too large.

[dfstio]

In case of Merkle Tree, the json file might be too large.

I agree; the files are quite large. You can make them smaller by using radix 36 and then zipping the result:

  toCompressedJSON() {
    const nodes: { [key: string]: string } = {};
    for (const level in this.nodes) {
      const node: string[] = [];
      for (const index in this.nodes[level]) {
        node.push(BigInt(index).toString(36));
        node.push(this.nodes[level][index].toBigInt().toString(36));
      }
      nodes[level] = node.join(".");
    }
    return {
      height: this.height,
      nodes,
    };
  }

  static fromCompressedJSON(json: any) {
    function convert(value: string, radix: number) {
      return [...value.toString()].reduce(
        (r, v) => r * BigInt(radix) + BigInt(parseInt(v, radix)),
        0n
      );
    }
    const tree = new MerkleTree(json.height);
    for (const level in json.nodes) {
      const node = json.nodes[level].split(".");
      for (let i = 0; i < node.length; i += 2) {
        tree.setNode(
          parseInt(level),
          BigInt(convert(node[i], 36)),
          Field(BigInt(convert(node[i + 1], 36)))
        );
      }
    }
    return tree;
  }

For the map with 10,000 elements, the file size is 188M unzipped and 128M zipped. Recreating a map with 10,000 elements from entries takes 9 min 15 sec; from a zip file, 1 min 17 sec, there is an 8 min difference.