JSON corruption due concurrent file write

[vird]

trafficstars

Due internal actions of edge wallet it can corrupt WalletName.json sample of corrupted file {"encryptionType":0,"iv_hex":"KmI7"}nbccfS5KIQIGo9k2iPW3Zp8pxV2Zg4XGo5sG3qHCU="}

You can look at really similar problem here https://github.com/nodejs/node/issues/7978 Problem is located here https://github.com/EdgeApp/edge-core-js/blob/master/src/core/storage/repo.js#L78 We can call saveChanges before other saveChanges is occurs and now disklet comes... It has relatively slow file write https://github.com/EdgeApp/disklet/blob/master/src/backends/node.ts#L89

So it tries to write, can't, tries to make directory and write one more time. That's enough for fast second saveChanges call. So first deepWriteFile made directory and tries to write second saveChanges call calls deepWriteFile and tries to write too.

[vird]

Proof.

/**
 * This will save a change-set into the local storage.
 * This function ignores folder-level deletes and overwrites,
 * but those can't happen under the current rules anyhow.
 */
var fireCount = 0;
export function saveChanges(
	disklet,
	changes
) {
	if (fireCount) {
		console.log("FIRE!!!");
		process.exit(1);
	}
	fireCount++;
	return Promise.all(
		Object.keys(changes).map(path => {
			const json = changes[path]
			return json != null
				? disklet.setText(path, JSON.stringify(json))
				: disklet.delete(path)
		})
	).then(()=>{
		fireCount--;
	})
}

Note I've removed some probably condidential info from screenshot

[vird]

Also I've found that up to 4 concurrent saveChanges calls was in progress when 5th started.

if (fireCount > 0) {
    console.log("FIRE!!!", fireCount);
}

[vird]

fix proposal https://github.com/vird/edge-core-js/commit/887b5bc310630622c7eb58cc7cbfff75f99a411d Note not MR yet, because I didn't get what flow errors means :-(

[swansontec]

Yikes! Good catch.

If the Disklet node.js backend really has this problem, then it would affect other areas like login and blockchain syncing, as well. I think the proper fix needs to happen in Disklet itself, rather than the sync module, since that will fix everything at once.