`hash_plutus_data` returning a new hash for the same `PlutusData` instance - `11.0.0-rc.6`

[abdelkrimdev]

After upgrading to the latest version of the CSL 11.0.0-rc.6 I noticed that I'm getting a new hash when calling the hash_plutus_data function which is breaking the backward compatibility and make it impossible to unlock old assets from the script

[SebastienGllmt]

If possible, it would be nice if you could post the data, post the hash you get and also post what you think the expected hash should be

[abdelkrimdev]

If possible, it would be nice if you could post the data, post the hash you get and also post what you think the expected hash should be

Yes of course!

RAW Datum:

d8799fd8799f581ca183bf86925f66c579a3745c9517744399679b090927b8f6e2f2e1bb4f616461706541696c656e416d61746fffd8799f581c9a4e855293a0b9af5e50935a331d83e7982ab5b738ea0e6fc0f9e6564e4652414d455f36353030335f4c30ff581cbea1c521df58f4eeef60c647e5ebd88c6039915409f9fd6454a476b9ff

expected hash: a7b8bbb79ffac027a596082a3510f9c17e09996ff76844e8c4461175e1cf06ff

new hash: ec3028f46325b983a470893a8bdc1b4a100695b635fb1237d301c3490b23e89b

[vsubhuman]

@abdelkrimdev , I am getting hash ec3028f46325b983a470893a8bdc1b4a100695b635fb1237d301c3490b23e89b for that datum in version 10.2 of the library as well.

If you have got a hash a7b8bbb79ffac027a596082a3510f9c17e09996ff76844e8c4461175e1cf06ff then there possibly some mistake. Make sure you datum matches exactly what has been used to create the output.

[abdelkrimdev]

@abdelkrimdev , I am getting hash ec3028f46325b983a470893a8bdc1b4a100695b635fb1237d301c3490b23e89b for that datum in version 10.2 of the library as well.

If you have got a hash a7b8bbb79ffac027a596082a3510f9c17e09996ff76844e8c4461175e1cf06ff then there possibly some mistake. Make sure you datum matches exactly what has been used to create the output.

Hey @vsubhuman 😄 the CBOR is the same the only thing I changed is the serialization lib version.

9.1.0: --> a7b8bbb79ffac027a596082a3510f9c17e09996ff76844e8c4461175e1cf06ff 11.0.0-rc.6: --> ec3028f46325b983a470893a8bdc1b4a100695b635fb1237d301c3490b23e89b

[vsubhuman]

@abdelkrimdev, ah yes, if you mean a version before 10.0 then you are right.

Still, though, in the version 9.1.0 by parsing exactly the CBOR provided by you I am getting hash 816cdf6d4d8cba3ad0188ca643db95ddf0e03cdfc0e75a9550a72a82cb146222. No idea how did you manage to get the value you mention as expected.

This code snippet:

let pdata = PlutusData::from_bytes(hex::decode("d8799fd8799f581ca183bf86925f66c579a3745c9517744399679b090927b8f6e2f2e1bb4f616461706541696c656e416d61746fffd8799f581c9a4e855293a0b9af5e50935a331d83e7982ab5b738ea0e6fc0f9e6564e4652414d455f36353030335f4c30ff581cbea1c521df58f4eeef60c647e5ebd88c6039915409f9fd6454a476b9ff").unwrap()).unwrap();
println!(hex::encode(hash_plutus_data(&pdata).to_bytes()));

Produces 816cdf6d4d8cba3ad0188ca643db95ddf0e03cdfc0e75a9550a72a82cb146222 in version 9.1.0 and ec3028f46325b983a470893a8bdc1b4a100695b635fb1237d301c3490b23e89b in any version starting with 10.0.0.

This happens because the behaviour were incorrect before the version 10.0.0 and got fixed there. The CBOR you have provided has arrays serialised with indefinite length, which is not canonically correct, but that't how it's done in Cardano CLI:

In any version before 10.0.0 the library did not respect the original format the datum was deserialised from and when the hash_plutus_data been called the same datum have been re-serialised using arrays with definite length, which produces a different sequence of bytes and a different hash. In any version starting with 10.0.0 this behaviour is fixed and the library produces datums in a format matching Cardano CLI and also respects the original format of deserialised datums.

This means that you have been using a datum with indefinite arrays, but the hashes been created for datum with definite arrays. Which is a really unfortunate side-effect of using CBOR in general which allows same information to be encoded in multiple different forms.

You should be able to fix your issue by taking your datums and converting them to CBOR with definite arrays and then they will match the used hashes exactly. One of the ways to do it would be to take the library version 9.1.0 that you been using and running this on each datum:

bytesToHex(
    PlutusData.from_bytes(
        hexToBytes(datumHex)
    ).to_bytes()
)

Just parsing a datum from bytes and then putting them right back into bytes will produce the different CBOR for you, which will match the hashes that were incorrectly created in that library version.

For example, the datum that you have specified above is converted into this:

d87983d87982581ca183bf86925f66c579a3745c9517744399679b090927b8f6e2f2e1bb4f616461706541696c656e416d61746fd87982581c9a4e855293a0b9af5e50935a331d83e7982ab5b738ea0e6fc0f9e6564e4652414d455f36353030335f4c30581cbea1c521df58f4eeef60c647e5ebd88c6039915409f9fd6454a476b9

Which is the same exact information but with definite arrays:

And executing this code snippet:

let pdata = PlutusData::from_bytes(hex::decode("d87983d87982581ca183bf86925f66c579a3745c9517744399679b090927b8f6e2f2e1bb4f616461706541696c656e416d61746fd87982581c9a4e855293a0b9af5e50935a331d83e7982ab5b738ea0e6fc0f9e6564e4652414d455f36353030335f4c30581cbea1c521df58f4eeef60c647e5ebd88c6039915409f9fd6454a476b9").unwrap()).unwrap();
println!(hex::encode(hash_plutus_data(&pdata).to_bytes()));

Gives the same hash 816cdf6d4d8cba3ad0188ca643db95ddf0e03cdfc0e75a9550a72a82cb146222 in the latest version of the library.

[adamutils]

Can you guys let me know if my understanding is correct.

1. In cardano serialization lib 9.1.3 we need first do

    const breedingDatum = createBreedingDatum(pkh, breedingParams);
    const datum = await toPlutusData(breedingDatum);
    const datumHex = datum.to_hex()

    const compatibleDatumHex = bytesToHex(
        PlutusData.from_bytes(
            hexToBytes(datumHex)
        ).to_bytes()
    )

2. Next we switch to cardano-serialization-lib in version 11.0.x we should take compatibleDatumHex and use it in following code ?

    const redeemer = createRedeemer(0, RedeemerType.CloseAdmin);

    const latestSlot = await getLatestBlockSlot()
    txBuilder.set_validity_start_interval(latestSlot);

    const plutusWitness = PlutusWitness.new(scripts.get(0), PlutusData.from_hex(compatibleDatumHex), redeemer)
    txBuilder.add_plutus_script_input(
        plutusWitness,
        unspentOutput.scriptUtxo.input(),
        contractAssetOutput
    )

txBuilder.add_required_signer(baseAddress.payment_cred().to_keyhash() as Ed25519KeyHash)

    const txUnspentOutputs  = TransactionUnspentOutputs.new();
    const walletOutputs  = await cardano.getUtxos();
    walletOutputs.forEach(utxo => txUnspentOutputs.add(utxo))
    txBuilder.add_inputs_from(txUnspentOutputs, 2)
    
    const collateral = await getCollateralUnspentTransactionOutput(api)
    var txInputBuilder = TxInputsBuilder.new()
    collateral.forEach((utxo) => {
        txInputBuilder.add_input(utxo.output().address(), utxo.input(), utxo.output().amount())
    });
    txBuilder.set_collateral(txInputBuilder)
    
    const costModels = TxBuilderConstants.plutus_alonzo_cost_models();
    txBuilder.calc_script_data_hash(costModels)
    
    txBuilder.add_change_if_needed(selfAddress)
    const tx = txBuilder.build_tx();

My assumption is that we should be using plutus_alonzo_cost_model because tokens were locked before vasil hardfork era. please correct me if i'm wrong