librustzcash fails to spend a note at the edge of a shard

[AArnott]

One of my eZcash users is getting an error from librustzcash:

"commitmenttree: unable to compute root; missing values for nodes [Address { level: Level(0), index: 73203711 }]"

This number is exactly -1 from a multiple of 2^16, suggesting the software has a bug in processing notes near the edge of a shard.

See discord discussion: https://discord.com/channels/809218587167293450/1300986651484360716

[daira]

What commit of librustzcash is being used?

[AArnott]

That was on 04478c791acfb4cb1015affd17f12565f88bcc43

[str4d]

I spent the past day building https://github.com/zcash/zcash-devtool/pull/76 to enable debugging this kind of issue. In a wallet I have access to, I observe a level 1 nil node, indicating that the ShardTree contains neither the two leaf nodes nor their parent, and has been pruned back to level 2. The nil node is on the right-most side of a subtree, same as in this issue.

[str4d]

That was on 04478c7

This is zcash_client_sqlite 0.11.2 plus whatever was in main after it was merged back, in particular #1524. I don't see any changes between zcash_client_sqlite 0.11.0 and that commit related to shardtree, so I think that means it was using shardtree 0.4 (which we upgraded to for zcash_client_sqlite 0.11.0 in #1493). That was the bugfix release in which we supposedly fixed the pruning-related bugs.