Compiler: remove unused data constructors

[kk-hainq]

This is an ongoing attempt to reduce script size compiling from Haskell. It pursuits #4148 and maybe #4147 as well. Note that this is highly experimental and needs many more tests and benchmarks. Until proven, any state of this branch might produce the wrong code. I'm already aware of a few test cases that seem wrong.

Also want to write a mega-thread on reducing script size for others to join as well.

Here are the initial stats on plutus-apps/plutus-use-cases@f5f7adce04c161c3e37f095cf6af4a39efe4a31b. Few notes:

• Most dApp developers are only interested in the final on-chain format?
• The separation in compilation between several functions still scares me. Will double-check and fix/improve the compilation interface if required.

Name	Before	After	Improvement
Plutus.Contracts.Auction.validate.pir-initial.flat	97630	97630	0.00%
Plutus.Contracts.Auction.validate.pir-simplified.flat	73662	73363	0.41%
Plutus.Contracts.Auction.validate.plc.flat	149443	145513	2.63%
Plutus.Contracts.Auction.validate.uplc.flat	33426	31766	4.97%
Plutus.Contracts.Auction.wrap.pir-initial.flat	27239	27239	0.00%
Plutus.Contracts.Auction.wrap.pir-simplified.flat	13593	13593	0.00%
Plutus.Contracts.Auction.wrap.plc.flat	35824	35824	0.00%
Plutus.Contracts.Auction.wrap.uplc.flat	12805	12805	0.00%
Plutus.Contracts.Crowdfunding.validate.pir-initial.flat	23857	23857	0.00%
Plutus.Contracts.Crowdfunding.validate.pir-simplified.flat	15153	14097	6.97%
Plutus.Contracts.Crowdfunding.validate.plc.flat	35392	26021	26.48%
Plutus.Contracts.Crowdfunding.validate.uplc.flat	10471	7397	29.36%
Plutus.Contracts.Crowdfunding.wrap.pir-initial.flat	25439	25439	0.00%
Plutus.Contracts.Crowdfunding.wrap.pir-simplified.flat	12729	12729	0.00%
Plutus.Contracts.Crowdfunding.wrap.plc.flat	33515	33515	0.00%
Plutus.Contracts.Crowdfunding.wrap.uplc.flat	11997	11997	0.00%
Plutus.Contracts.Currency.policy.pir-initial.flat	46894	46894	0.00%
Plutus.Contracts.Currency.policy.pir-simplified.flat	27174	27174	0.00%
Plutus.Contracts.Currency.policy.plc.flat	52843	52843	0.00%
Plutus.Contracts.Currency.policy.uplc.flat	17546	17546	0.00%
Plutus.Contracts.Escrow.validate.pir-initial.flat	50761	50761	0.00%
Plutus.Contracts.Escrow.validate.pir-simplified.flat	31928	31270	2.06%
Plutus.Contracts.Escrow.validate.plc.flat	62753	55502	11.55%
Plutus.Contracts.Escrow.validate.uplc.flat	15404	12874	16.42%
Plutus.Contracts.Escrow.wrap.pir-initial.flat	25246	25246	0.00%
Plutus.Contracts.Escrow.wrap.pir-simplified.flat	12663	12663	0.00%
Plutus.Contracts.Escrow.wrap.plc.flat	33294	33294	0.00%
Plutus.Contracts.Escrow.wrap.uplc.flat	11984	11984	0.00%
Plutus.Contracts.GameStateMachine.validate.pir-initial.flat	96193	96193	0.00%
Plutus.Contracts.GameStateMachine.validate.pir-simplified.flat	71442	71143	0.42%
Plutus.Contracts.GameStateMachine.validate.plc.flat	146433	142503	2.68%
Plutus.Contracts.GameStateMachine.validate.uplc.flat	32504	30852	5.08%
Plutus.Contracts.GameStateMachine.wrap.pir-initial.flat	27476	27476	0.00%
Plutus.Contracts.GameStateMachine.wrap.pir-simplified.flat	13779	13779	0.00%
Plutus.Contracts.GameStateMachine.wrap.plc.flat	36386	36386	0.00%
Plutus.Contracts.GameStateMachine.wrap.uplc.flat	12966	12966	0.00%
Plutus.Contracts.Governance.validate.pir-initial.flat	115870	115870	0.00%
Plutus.Contracts.Governance.validate.pir-simplified.flat	88271	87972	0.34%
Plutus.Contracts.Governance.validate.plc.flat	165874	161944	2.37%
Plutus.Contracts.Governance.validate.uplc.flat	38765	37100	4.30%
Plutus.Contracts.Governance.wrap.pir-initial.flat	28609	28609	0.00%
Plutus.Contracts.Governance.wrap.pir-simplified.flat	14337	14337	0.00%
Plutus.Contracts.Governance.wrap.plc.flat	38058	38058	0.00%
Plutus.Contracts.Governance.wrap.uplc.flat	13752	13752	0.00%
Plutus.Contracts.MultiSig.validate.pir-initial.flat	11367	11367	0.00%
Plutus.Contracts.MultiSig.validate.pir-simplified.flat	5944	4446	25.20%
Plutus.Contracts.MultiSig.validate.plc.flat	24286	12162	49.92%
Plutus.Contracts.MultiSig.validate.uplc.flat	6502	2798	56.97%
Plutus.Contracts.MultiSig.wrap.pir-initial.flat	25070	25070	0.00%
Plutus.Contracts.MultiSig.wrap.pir-simplified.flat	12743	12743	0.00%
Plutus.Contracts.MultiSig.wrap.plc.flat	32986	32986	0.00%
Plutus.Contracts.MultiSig.wrap.uplc.flat	11941	11941	0.00%
Plutus.Contracts.MultiSigStateMachine.validate.pir-initial.flat	104496	104496	0.00%
Plutus.Contracts.MultiSigStateMachine.validate.pir-simplified.flat	77121	76822	0.39%
Plutus.Contracts.MultiSigStateMachine.validate.plc.flat	153822	149892	2.55%
Plutus.Contracts.MultiSigStateMachine.validate.uplc.flat	35038	33387	4.71%
Plutus.Contracts.MultiSigStateMachine.wrap.pir-initial.flat	27766	27766	0.00%
Plutus.Contracts.MultiSigStateMachine.wrap.pir-simplified.flat	13978	13978	0.00%
Plutus.Contracts.MultiSigStateMachine.wrap.plc.flat	37206	37206	0.00%
Plutus.Contracts.MultiSigStateMachine.wrap.uplc.flat	13410	13410	0.00%
Plutus.Contracts.PingPong.validate.pir-initial.flat	92315	92315	0.00%
Plutus.Contracts.PingPong.validate.pir-simplified.flat	69557	69258	0.43%
Plutus.Contracts.PingPong.validate.plc.flat	143428	139498	2.74%
Plutus.Contracts.PingPong.validate.uplc.flat	31562	29916	5.22%
Plutus.Contracts.PingPong.wrap.pir-initial.flat	26226	26226	0.00%
Plutus.Contracts.PingPong.wrap.pir-simplified.flat	13157	13157	0.00%
Plutus.Contracts.PingPong.wrap.plc.flat	34887	34887	0.00%
Plutus.Contracts.PingPong.wrap.uplc.flat	12526	12526	0.00%
Plutus.Contracts.Prism.STO.policy.pir-initial.flat	48744	48744	0.00%
Plutus.Contracts.Prism.STO.policy.pir-simplified.flat	29190	29190	0.00%
Plutus.Contracts.Prism.STO.policy.plc.flat	55545	55545	0.00%
Plutus.Contracts.Prism.STO.policy.uplc.flat	17482	17482	0.00%
Plutus.Contracts.PubKey.validate.pir-initial.flat	7901	7901	0.00%
Plutus.Contracts.PubKey.validate.pir-simplified.flat	4409	2911	33.98%
Plutus.Contracts.PubKey.validate.plc.flat	21929	9805	55.29%
Plutus.Contracts.PubKey.validate.uplc.flat	5669	1965	65.34%
Plutus.Contracts.PubKey.wrap.pir-initial.flat	25070	25070	0.00%
Plutus.Contracts.PubKey.wrap.pir-simplified.flat	12743	12743	0.00%
Plutus.Contracts.PubKey.wrap.plc.flat	32986	32986	0.00%
Plutus.Contracts.PubKey.wrap.uplc.flat	11941	11941	0.00%
Plutus.Contracts.SealedBidAuction.validate.pir-initial.flat	108309	108309	0.00%
Plutus.Contracts.SealedBidAuction.validate.pir-simplified.flat	81103	80804	0.37%
Plutus.Contracts.SealedBidAuction.validate.plc.flat	159265	155335	2.47%
Plutus.Contracts.SealedBidAuction.validate.uplc.flat	37182	35537	4.42%
Plutus.Contracts.SealedBidAuction.wrap.pir-initial.flat	28583	28583	0.00%
Plutus.Contracts.SealedBidAuction.wrap.pir-simplified.flat	14421	14421	0.00%
Plutus.Contracts.SealedBidAuction.wrap.plc.flat	38281	38281	0.00%
Plutus.Contracts.SealedBidAuction.wrap.uplc.flat	13687	13687	0.00%
Plutus.Contracts.SimpleEscrow.validate.pir-initial.flat	47753	47753	0.00%
Plutus.Contracts.SimpleEscrow.validate.pir-simplified.flat	28117	27459	2.34%
Plutus.Contracts.SimpleEscrow.validate.plc.flat	57755	50504	12.55%
Plutus.Contracts.SimpleEscrow.validate.uplc.flat	14482	11952	17.47%
Plutus.Contracts.SimpleEscrow.wrap.pir-initial.flat	26727	26727	0.00%
Plutus.Contracts.SimpleEscrow.wrap.pir-simplified.flat	13462	13462	0.00%
Plutus.Contracts.SimpleEscrow.wrap.plc.flat	35713	35713	0.00%
Plutus.Contracts.SimpleEscrow.wrap.uplc.flat	12484	12484	0.00%
Plutus.Contracts.Stablecoin.validate.pir-initial.flat	155057	155057	0.00%
Plutus.Contracts.Stablecoin.validate.pir-simplified.flat	109901	109496	0.37%
Plutus.Contracts.Stablecoin.validate.plc.flat	198010	192882	2.59%
Plutus.Contracts.Stablecoin.validate.uplc.flat	50144	48039	4.20%
Plutus.Contracts.Stablecoin.wrap.pir-initial.flat	31942	31942	0.00%
Plutus.Contracts.Stablecoin.wrap.pir-simplified.flat	15549	15549	0.00%
Plutus.Contracts.Stablecoin.wrap.plc.flat	39793	39793	0.00%
Plutus.Contracts.Stablecoin.wrap.uplc.flat	14221	14221	0.00%
Plutus.Contracts.Swap.validate.pir-initial.flat	126572	126572	0.00%
Plutus.Contracts.Swap.validate.pir-simplified.flat	84865	84807	0.07%
Plutus.Contracts.Swap.validate.plc.flat	165782	165111	0.40%
Plutus.Contracts.Swap.validate.uplc.flat	43753	43492	0.60%
Plutus.Contracts.TokenAccount.validate.pir-initial.flat	21848	21848	0.00%
Plutus.Contracts.TokenAccount.validate.pir-simplified.flat	14509	13249	8.68%
Plutus.Contracts.TokenAccount.validate.plc.flat	35663	24957	30.02%
Plutus.Contracts.TokenAccount.validate.uplc.flat	9241	5722	38.08%
Plutus.Contracts.TokenAccount.wrap.pir-initial.flat	25070	25070	0.00%
Plutus.Contracts.TokenAccount.wrap.pir-simplified.flat	12743	12743	0.00%
Plutus.Contracts.TokenAccount.wrap.plc.flat	32986	32986	0.00%
Plutus.Contracts.TokenAccount.wrap.uplc.flat	11941	11941	0.00%
Plutus.Contracts.Prism.Credential.policy.pir-initial.flat	28153	28153	0.00%
Plutus.Contracts.Prism.Credential.policy.pir-simplified.flat	14255	14255	0.00%
Plutus.Contracts.Prism.Credential.policy.plc.flat	35342	35342	0.00%
Plutus.Contracts.Prism.Credential.policy.uplc.flat	12587	12587	0.00%
Plutus.Contracts.Prism.StateMachine.validate.pir-initial.flat	95666	95666	0.00%
Plutus.Contracts.Prism.StateMachine.validate.pir-simplified.flat	71877	71578	0.42%
Plutus.Contracts.Prism.StateMachine.validate.plc.flat	147335	143405	2.67%
Plutus.Contracts.Prism.StateMachine.validate.uplc.flat	32722	31076	5.03%
Plutus.Contracts.Prism.StateMachine.wrap.pir-initial.flat	25938	25938	0.00%
Plutus.Contracts.Prism.StateMachine.wrap.pir-simplified.flat	13015	13015	0.00%
Plutus.Contracts.Prism.StateMachine.wrap.plc.flat	34218	34218	0.00%
Plutus.Contracts.Prism.StateMachine.wrap.uplc.flat	12378	12378	0.00%
Plutus.Contracts.Future.validate.pir-initial.flat	131721	131721	0.00%
Plutus.Contracts.Future.validate.pir-simplified.flat	98750	98345	0.41%
Plutus.Contracts.Future.validate.plc.flat	186693	181565	2.75%
Plutus.Contracts.Future.validate.uplc.flat	40072	37993	5.19%
Plutus.Contracts.Future.wrap.pir-initial.flat	32081	32081	0.00%
Plutus.Contracts.Future.wrap.pir-simplified.flat	16495	16495	0.00%
Plutus.Contracts.Future.wrap.plc.flat	44024	44024	0.00%
Plutus.Contracts.Future.wrap.uplc.flat	14415	14415	0.00%
Plutus.Contracts.Uniswap.OffChain.validate.pir-initial.flat	32773	32773	0.00%
Plutus.Contracts.Uniswap.OffChain.validate.pir-simplified.flat	16683	16643	0.24%
Plutus.Contracts.Uniswap.OffChain.validate.plc.flat	37848	37606	0.64%
Plutus.Contracts.Uniswap.OffChain.validate.uplc.flat	13346	13270	0.57%
Plutus.Contracts.Uniswap.OffChain.wrap.pir-initial.flat	137542	137542	0.00%
Plutus.Contracts.Uniswap.OffChain.wrap.pir-simplified.flat	88092	87287	0.91%
Plutus.Contracts.Uniswap.OffChain.wrap.plc.flat	125212	118164	5.63%
Plutus.Contracts.Uniswap.OffChain.wrap.uplc.flat	36898	34520	6.44%
Plutus.Contracts.Uniswap.OffChain.policy.pir-initial.flat	31183	31183	0.00%
Plutus.Contracts.Uniswap.OffChain.policy.pir-simplified.flat	15433	15433	0.00%
Plutus.Contracts.Uniswap.OffChain.policy.plc.flat	39994	39994	0.00%
Plutus.Contracts.Uniswap.OffChain.policy.uplc.flat	14244	14244	0.00%
Plutus.Contracts.Vesting.validate.pir-initial.flat	46241	46241	0.00%
Plutus.Contracts.Vesting.validate.pir-simplified.flat	32085	31709	1.17%
Plutus.Contracts.Vesting.validate.plc.flat	62228	57529	7.55%
Plutus.Contracts.Vesting.validate.uplc.flat	15920	14065	11.65%
Plutus.Contracts.Vesting.wrap.pir-initial.flat	25070	25070	0.00%
Plutus.Contracts.Vesting.wrap.pir-simplified.flat	12743	12743	0.00%
Plutus.Contracts.Vesting.wrap.plc.flat	32986	32986	0.00%
Plutus.Contracts.Vesting.wrap.uplc.flat	11941	11941	0.00%

Pre-submit checklist:

• Branch
  • [x] Tests are provided (if possible)
  • [x] Commit sequence broadly makes sense
  • [x] Key commits have useful messages
  • [x] Relevant tickets are mentioned in commit messages
  • [x] Formatting, materialized Nix files, PNG optimization, etc. are updated
• PR
  • [x] Self-reviewed the diff
  • [x] Useful pull request description
  • [ ] Reviewer requested

[michaelpj]

In the short term I suggest focussing on https://github.com/input-output-hk/plutus/issues/4147, which I'm pretty sure is sound.

I'd also suggest doing the changes to dependency analysis in a standalone PR, we should be able to merge that separately and get it out of the way.

[michaelpj]

Hmm, thinking about it more, perhaps we should leave the code in, just behind a compiler flag that's off by default. Plausibly the user can tell us whether truly-global optimizations are safe or not (i.e. if they're not using Lift).

[kk-hainq]

In the short term I suggest focussing on #4147, which I'm pretty sure is sound.

I'd also suggest doing the changes to dependency analysis in a standalone PR, we should be able to merge that separately and get it out of the way.

Yeah, this current attempt is pretty messy and was more of a learning experience for me. I'll lay things out in a sane order with modular milestones and post a mega-issue for you and others to review. Hopefully this Wednesday and definitely before the weekend.

Hmm, thinking about it more, perhaps we should leave the code in, just behind a compiler flag that's off by default. Plausibly the user can tell us whether truly-global optimizations are safe or not (i.e. if they're not using Lift).

Yeah I've already added a flag in poTruncateTypes -> coTruncateTypes.

-- This options allows the compiler to remove unsused constructors when removing dead bindings.
-- Which might not be desirable when terms are compiled then applied separately.
-- Which yields incompatible types when certain constructors are used in a few programs but not in others.
-- This is usually useful when compiling large standalone programs like validator scripts.
, poTruncateTypes                  :: Bool

Have also added a StableTerms module to the test suite.

-- We don't truncate types for terms that are compiled separately, to be applied afterward in other test cases.
{-# OPTIONS_GHC -fplugin-opt PlutusTx.Plugin:no-truncate-types #-}

module Plugin.Data.StableTerms where

To keep the full types for:

stableListConstruct :: CompiledCode [Integer]
stableListConstruct = plc (Proxy @"stableListConstruct") ([]::[Integer])

stableTrue :: CompiledCode Bool
stableTrue = plc (Proxy @"stableTrue") True

stableFalse :: CompiledCode Bool
stableFalse = plc (Proxy @"stableFalse") False

To pass relevant existing tests.

Have to turn it off in Lift as well:

-- Also don't truncate types for now as it's usually preferred to keep lifted terms' types in tact.
-- TODO: Optimize the whole lifting process
let ccConfig = set (ccOpts . coMaxSimplifierIterations) 0 $
               set (ccOpts . coTruncateTypes) False $
               (toDefaultCompilationCtx tcConfig)

And yeah, I set it on by default for now and only disable it on StableTerms to make development faster. The true solution must have it off by default and re-organize the test suite accordingly.