plutus
plutus copied to clipboard
Some benchmarks are screwed
#5867 fixed some of the screwed up benchmarks (See this comment for an explanation of what went wrong), but we still have more of these issues in
budgeting-benchmarlowebls12-381-costs- potentially more
These need to be fixed as well. It would also be great to try to ensure we don't run into this issue again, although I'm not sure if it's possible.
I did this for bls12-381-benchmarks, marlowe, and cek-calibration in #5905. However
- The Marlowe benchmarks showed a speedup of 10-20%. It might be instructive to find out why this is. The speedup seems to be inversely correlated with the size of the script: the largest scripts have the smallest speedup. The relationship seems quite weak though, and we don't have a big enough selection of different script sizes to come to a firm conclusion.
- The validation benchmarks initially slowed down by 2%, despite the fact that they weren't changed. This seems to depend on the order and number of exports from
PlutusBenchmark.Common: see #5906. Trial and error led to an arrangement of the exports that got the validation times back to where they were, but that's not a reliable way of doing things.