cardano-ledger icon indicating copy to clipboard operation
cardano-ledger copied to clipboard

Added first Necessary And Sufficient EraRule tests

Open TimSheard opened this issue 1 year ago • 2 comments

Adds a new style of tests.

One writes a specification for a triple (Environment, State, Signal). This specifucation specifies the minimal constraints to make the tests pass. Tests can be run in 3 Modes 1) Apply, Invert, Remove In Apply Mode, we test that the given tests are Sufficient to pass the STS rules. in Invert Mode, we test that the given tests are Neccssary, as we invert them, and then Expect the test to Fail. if the test doesn't fail, the test is Not necessary. in Remove Mode, we remove the constraint. It may or may not pass, as one can get lucky. We use classify to record which constraints can get lucky.

Checklist

  • [x] Commit sequence broadly makes sense and commits have useful messages
  • [x] New tests are added if needed and existing tests are updated
  • [ ] When applicable, versions are updated in .cabal and CHANGELOG.md files according to the versioning process.
  • [ ] The version bounds in .cabal files for all affected packages are updated. If you change the bounds in a cabal file, that package itself must have a version increase. (See RELEASING.md)
  • [ ] All visible changes are prepended to the latest section of a CHANGELOG.md for the affected packages. New section is never added with the code changes. (See RELEASING.md)
  • [x] Code is formatted with fourmolu (use scripts/fourmolize.sh)
  • [x] Cabal files are formatted (use scripts/cabal-format.sh)
  • [x] hie.yaml has been updated (use scripts/gen-hie.sh)
  • [x] Self-reviewed the diff

TimSheard avatar Jun 18 '24 19:06 TimSheard

I haven't had time yet to look in deep detail. However, what I'm a little bit concerned about here is that assertWith takes only a Status. The problem with that as I see it is that it means we alter all calls to assertWith with the same status flag. I would suggest doing something like this:

data Status a = Status { alterations :: Map a Alteration }
data Alteration = Apply | Invert | Remove
assertWith :: Alteration -> Term fn Bool -> Pred fn
assertWith Apply = assert
assertWith Remove = const TruePred
assertWith Invert = assert . not_

assertWithTag :: (IsBaseUniverse fn, Ord a)=> a -> Status a -> Term fn Bool -> Pred fn
assertWithTag tag status = assertWith (Map.lookupWithDefault Apply tag $ alterations status)

This way it would be possible to control the behaviour of different assertions in a neat way given we write some convenient syntax for constructing Status. Then comes the question of what type to use for the tag, we could simply do Status String for most use cases, or one could introduce a little enum type for one's specs.

MaximilianAlgehed avatar Jun 19 '24 06:06 MaximilianAlgehed

I completely agree with this. One thing missing is communicating, back to the test, what extra information is being used. That will be necessary to make good error messages if one of them fails.

TimSheard avatar Jun 19 '24 11:06 TimSheard

Closing this as something that we no longer want to proceed.

lehins avatar Dec 10 '24 21:12 lehins