Look further at: MissingScriptWitnessesUTXOW (fromList [])

Open TimSheard opened this issue 3 years ago • 0 comments

In https://github.com/input-output-hk/cardano-ledger/pull/3039 we fixed the seemingly random MissingScriptWitnessesUTXOW (fromList []) failure, by discarding traces where there was an unnecessary script witness.

It would be better to not generate transactions with these extra script witnesses. But this seems quite difficult to do, as generating witnesses is intricately tied up with the fix-point calculation. Here are some notes that might be useful. There is some evidence that this arises because the script hash appears as the PolicyId in a Value. At least that is the case in one failing trace. In that trace here is the transaction

ValidatedTx {
body = TxBodyConstr TxBodyRaw {
   _inputs = fromList
     [TxIn (TxId {_unTxId = SafeHash "7887389959cd0b37c21116c1658b195112605df6e1e7165921192b416eccaf32"}) (TxIx 1),
      TxIn (TxId {_unTxId = SafeHash "87a0cb55e115848fc0486f17a3d7c68d3fe2f3b24fc0c3cd32dae24459b8526e"}) (TxIx 0)],
   _collateral = fromList
     [TxIn (TxId {_unTxId = SafeHash "87a0cb55e115848fc0486f17a3d7c68d3fe2f3b24fc0c3cd32dae24459b8526e"}) (TxIx 0)],
   _outputs = StrictSeq {fromStrict = fromList
      [(Addr Testnet (KeyHashObj (KeyHash "963f3c80fd6cb6104d6cd5e114661fc0d7e9dcb9385acb996986c71c"))
                     (StakeRefBase (KeyHashObj (KeyHash "bd455eb0e7847df7c1e5552cad84621355b94f69cb3999832b54af10"))),
                     Value 4342 (fromList
                          [(PolicyID {policyID = ScriptHash "42c7a014a4cd5537f64e5ae8ec7349db3d8603e16765dc37f8fb6e67"},  ####### Note PolicyID is the extra script
                                             fromList [(79656c6c6f7730,560729),(79656c6c6f7733,477818),(79656c6c6f7735,276901)])]),SNothing),
      (Addr Testnet (KeyHashObj (KeyHash "094d78230eb61aaf1ff09f43ff9d3ce63b7932d499d5131757ee62dd"))
                    (StakeRefBase (KeyHashObj (KeyHash "3bd2ccfdb4fec76fc4f35a6da648a5ffcc4082c6642c64f38de2ea22"))),
                    Value 4537741 (fromList []),SNothing)]},
   _certs = StrictSeq {fromStrict = fromList
       [DCertDeleg (RegKey (KeyHashObj (KeyHash "e459931b2628c08350543b7e5202acd8cee0941d51ee1b08b42e6d70"))),
        DCertDeleg (Delegate (Delegation {_delegator = KeyHashObj (KeyHash "d840b13dcfcb99c5ad3fb54f0d007c335ed050c17f07b995458b2827"),
                                          _delegatee = KeyHash "15b7af45e7293a42b8eabb19bb0fd568116581a8b99fceaffb3e4bab"}))]},
  _wdrls = Wdrl {unWdrl = fromList
      [(RewardAcnt {getRwdNetwork = Testnet, getRwdCred = ScriptHashObj
                     (ScriptHash "395e0b6c308dbdfd6e41354b68f833b96990ecd93721699ed90a2113")},Coin 0),
       (RewardAcnt {getRwdNetwork = Testnet, getRwdCred = ScriptHashObj
                     (ScriptHash "504fca9befba2683bcf5797db908b891ff8e9aacfb3ce633fe495b45")},Coin 0),
       (RewardAcnt {getRwdNetwork = Testnet, getRwdCred = ScriptHashObj
                     (ScriptHash "5c0a02a40d0b72ea023a6628116c3fd7ca00e853dde5f7c90606aeff")},Coin 0),
       (RewardAcnt {getRwdNetwork = Testnet, getRwdCred = ScriptHashObj
                     (ScriptHash "64abdacc0e1cc6b7dcbcf601c28f3e882f5453dca4eb713591635f90")},Coin 0),
       (RewardAcnt {getRwdNetwork = Testnet, getRwdCred = ScriptHashObj
                     (ScriptHash "7c06fcc873546a676c9b83947385ccf5101d2320456ec3f2dd8179f9")},Coin 0),
       (RewardAcnt {getRwdNetwork = Testnet, getRwdCred = ScriptHashObj
                     (ScriptHash "9573fdfbcf7e2fdec139e2fcf7215883855933c9628928d5e5c0f3b4")},Coin 0)]},
  _txfee = Coin 1561575,
  _vldt = ValidityInterval {invalidBefore = SJust (SlotNo 536), invalidHereafter = SJust (SlotNo 537)},
  _update = SNothing,
  _reqSignerHashes = fromList [],
  _mint = Value 0 (fromList []),
  _scriptIntegrityHash = SNothing,
  _adHash = SNothing,
  _txnetworkid = SNothing},

  wits = TxWitnessRaw {
    _txwitsVKey = fromList [...]
    _txwitsBoot = fromList [],
    _txscripts = fromList
      [(ScriptHash "395e0b6c308dbdfd6e41354b68f833b96990ecd93721699ed90a2113",
          TimelockScript TimelockConstr Signature (KeyHash "380f0d3b9106f8e7e0ee50572044da97257a9d1c55f858f0ae1c9e33")),
       EXTRANEOUS  Comes from PolicyID of a Value with (Coin 4342) ... 
       (ScriptHash "42c7a014a4cd5537f64e5ae8ec7349db3d8603e16765dc37f8fb6e67",
          TimelockScript TimelockConstr AllOf (StrictSeq {fromStrict = fromList
            [TimelockConstr AllOf (StrictSeq {fromStrict = fromList [TimelockConstr AllOf (StrictSeq {fromStrict = fromList []})]})]})),
       (ScriptHash "504fca9befba2683bcf5797db908b891ff8e9aacfb3ce633fe495b45",
          TimelockScript TimelockConstr MOfN 3 (StrictSeq {fromStrict = fromList
             [TimelockConstr AnyOf (StrictSeq {fromStrict = fromList
                    [TimelockConstr TimeStart (SlotNo 26),
                     TimelockConstr TimeExpire (SlotNo 26)]}),
                     TimelockConstr Signature (KeyHash "3a3c9207e9d6a4f55c9be45e2279d3d9826845e01573d8248fcde754"),
                     TimelockConstr Signature (KeyHash "2508e7e5b012b1af2b9942dc6837d68d38ac6f7fcd0261c08b444d4d")]})),
       (ScriptHash "5c0a02a40d0b72ea023a6628116c3fd7ca00e853dde5f7c90606aeff",
           TimelockScript TimelockConstr AnyOf (StrictSeq {fromStrict = fromList
              [TimelockConstr TimeStart (SlotNo 26),
              TimelockConstr TimeExpire (SlotNo 26)]})),
       (ScriptHash "64abdacc0e1cc6b7dcbcf601c28f3e882f5453dca4eb713591635f90",
           TimelockScript TimelockConstr AnyOf (StrictSeq {fromStrict = fromList
              [TimelockConstr Signature (KeyHash "2508e7e5b012b1af2b9942dc6837d68d38ac6f7fcd0261c08b444d4d"),
               TimelockConstr Signature (KeyHash "3a3c9207e9d6a4f55c9be45e2279d3d9826845e01573d8248fcde754"),
               TimelockConstr AnyOf (StrictSeq {fromStrict = fromList
                  [TimelockConstr TimeStart (SlotNo 26),
                   TimelockConstr TimeExpire (SlotNo 26)]})]})),
       (ScriptHash "7c06fcc873546a676c9b83947385ccf5101d2320456ec3f2dd8179f9",
           TimelockScript TimelockConstr Signature (KeyHash "d3b51656ce61098c59bef02b99529e3b16d9165951cf0c030f915602")),
       (ScriptHash "9573fdfbcf7e2fdec139e2fcf7215883855933c9628928d5e5c0f3b4",
           TimelockScript TimelockConstr Signature (KeyHash "7006429f96d5a5775d9bca2e69b6f69caeeb8fc34ef2df63763ce35c"))],
  _txdats = TxDatsRaw (fromList []),
  _txrdmrs = RedeemersRaw (fromList [])},
   isValid = IsValid True,
   auxiliaryData = SNothing}

Note the output, where the Value has a policy ID with script hash "42c7a014a4cd5537f64e5ae8ec7349db3d8603e16765dc37f8fb6e67" Note that, that is the script hash of the unneeded witness.

How does an extra witness lead to the MissingScriptWitnessesUTXOW (fromList []) failure? This arises because of this code in the Utxow rule in the alonzo era.

validateMissingScripts pp sNeeded sReceived =
  if HardForks.missingScriptsSymmetricDifference pp
    then
      sequenceA_
        [ failureUnless (sNeeded `Set.isSubsetOf` sReceived) $
            MissingScriptWitnessesUTXOW (sNeeded `Set.difference` sReceived),
          failureUnless (sReceived `Set.isSubsetOf` sNeeded) $
            ExtraneousScriptWitnessesUTXOW (sReceived `Set.difference` sNeeded)
        ]
    else
      failureUnless (sNeeded == sReceived) $
        MissingScriptWitnessesUTXOW (sNeeded `Set.difference` sReceived)  ### TRACE SHOWS WE ARE HERE

So he hardforks test is false, and we compute (sNeeded Set.difference sReceived) which turns out to be the empty set since sReceived is a superset of sNeeded. Here are the values of those variables.

sneeded =fromList
   [ScriptHash "395e0b6c308dbdfd6e41354b68f833b96990ecd93721699ed90a2113",
    ScriptHash "504fca9befba2683bcf5797db908b891ff8e9aacfb3ce633fe495b45",
    ScriptHash "5c0a02a40d0b72ea023a6628116c3fd7ca00e853dde5f7c90606aeff",
    ScriptHash "64abdacc0e1cc6b7dcbcf601c28f3e882f5453dca4eb713591635f90",
    ScriptHash "7c06fcc873546a676c9b83947385ccf5101d2320456ec3f2dd8179f9",
    ScriptHash "9573fdfbcf7e2fdec139e2fcf7215883855933c9628928d5e5c0f3b4"]
  received=fromList
   [ScriptHash "395e0b6c308dbdfd6e41354b68f833b96990ecd93721699ed90a2113",
    ScriptHash "42c7a014a4cd5537f64e5ae8ec7349db3d8603e16765dc37f8fb6e67",  -- Extra script hash
    ScriptHash "504fca9befba2683bcf5797db908b891ff8e9aacfb3ce633fe495b45",
    ScriptHash "5c0a02a40d0b72ea023a6628116c3fd7ca00e853dde5f7c90606aeff",
    ScriptHash "64abdacc0e1cc6b7dcbcf601c28f3e882f5453dca4eb713591635f90",
    ScriptHash "7c06fcc873546a676c9b83947385ccf5101d2320456ec3f2dd8179f9",
    ScriptHash "9573fdfbcf7e2fdec139e2fcf7215883855933c9628928d5e5c0f3b4"]

This explains why we get the error. But not why the extra witness is generated. But perhaps it is a usefull clue, that at least in this failure the hash comes from a PolicyID.

Once can reproduce this on commit 4fc32a6117db697bc928d1488d38e9e871e82dbe using this command TASTY_PATTERN='/total amount of Ada is preserved (Chain)/' cabal test --test-options="--quickcheck-replay=76070 "

Sep 16 '22 01:09 TimSheard