Develop one or more Pool Lifecycle proposals to become ZIPs.

[nathan-at-least]

Dropping this issue here as a discussion rallying point about "Pool Lifecycles". Brief overview:

The idea of "Pool Lifecycles" is to introduce a standard and consistent approach to phasing out older pools. By following a consistent transition between well-named states along some sort of expected timelines, users can anticipate and prepare for pool migrations better.

We've historically often only referred to funds stored in Sprout or Sapling addresses as "in pools" and we often might say "shielded pools". However, for this ticket, we use the term "pool" to include funds stored in t-addrs. In this conception all funds exist in exactly one pool at any time, and proposals need to account for the "transparent pool" lifecycle as well as existing "shielded pools" and any other kind of pool introduced in future upgrades.

The net benefits are:

• older pools present an ongoing security risk to the entire network, including all users and infrastructure operators (this is only partially and imperfectly mitigated by the turnstile design).
• older pools require an ongoing technical maintenance burden by developers, full nodes (including miners), and others.

By retiring them over time, users, developers, and the whole network generally benefits. However, it is important to note the class of users that pool retirement will impede:

• depending on the Pool Lifecycle design specifics, long term cold storage use cases may be impeded or prevented.

The obvious first candidate for pool retirement is Sprout, which was the first shielded pool, used two different proving systems so far in its lifetime, suffered from a known counterfeiting vulnerability with no yet-known compromise, and which is too inefficient to work on mobile shielded wallets.

Note there is a zcash community dev discord room for this topic.

Edit 2021-03-09: Dropped "shielded" in describing pools, then added paragraph redefining "pool" to encompass t-addr funds and to assert that all funds live in exactly one pool as a basic property of pools. (This should address https://github.com/zcash/zips/issues/452#issuecomment-789499087 )

[zookozcash]

Is there a specific reason why shielded pools are treated differently than the transparent pool in this? If the Zcash community doesn't apply the same lifecycle mechanics to the transparent pool as it does to shielded pools, that's a strong incentive for people to store their ZEC long-term in the transparent pool.

[nighthawk24]

Linking recent discussion on Zcash Community Forums https://forum.zcashcommunity.com/t/what-do-we-do-about-legacy-value-pools/37739/99

[daira]

Agreed that the transparent pool should be treated in the same way as shielded pools by this policy. It also has significant cryptographic risks, a high technical debt burden, and its existence complicates usability and privacy analysis.

[nathan-at-least]

Straw-Proposal "Unlikely Coast"

This is a straw-proposal I'm editing in place in this post. The codename came from a random word generator, in anticipation of alternative proposals.

Concepts

Pools

This proposal redefines the term "Pool" from any prior usage to improve precision and clarity.

All Zcash funds are associated with exactly one pool.

Pools are distinct in the protocol and the transaction functionality and technology stack between pools can (and does) vary. Pools are named based on the transaction feature codename which introduced the pool (ex: "Sprout Pool", "Sapling Pool"), plus "The Transparent Pool" comprised of Bitcoin-style UTXOs.

Any funds that transition between any pools can only do so through a Turnstile Mechanism which always publicly reveals the amount of this transition to account for money supply integrity. Note that it may not be possible to send funds into a pool when it is in certain stages of the Pool Lifecycle (see below).

This proposal anticipates that every pool has some limited lifetime, even if the limit is not defined or far in the future. By implication, this proposal precludes two currently-possible use cases and therefore has a potentially significant impact. See Use Cases below.

This proposal levies Legacy Support Fees (see below) and ultimately deletes pools from consensus node state, so this has significant material impact on all users and service/product designers.

Pool Non-Restrictions

This proposal specifically anticipates several possibilities around pools that have not yet occurred or may not be widely discussed:

• Pools may not always be strictly ordered in terms of chronology or "preference". For example a single Zcash network upgrade may introduce two pools with different features because that is the best design to achieve clear goals and balance trade-offs appropriately.
• Pools may not always use a UTXO/Note-like model. For example there may be "account-based" pools in the future.
• Pools may not be retired in chronological order, either due to up-front orderly planning or due to emergency protocol changes.

Turnstiles

Funds may only be transferred between pools through pool Turnstiles within a single atomic transaction. Each pool has both a single Entrance Turnstile and a single Exit Turnstile each with states that change through the pool's lifecycle (see below).

Funds may only pass through a turnstile by revealing the amount publicly. The current total Pool Balance of any pool is therefore publicly known through this Turnstile Accounting. The consensus rules must always prevent any pool balance from becoming negative by rejecting blocks in which this would occur, a rule known as Turnstile Enforcement. This design ensures that if there is any successful counterfeit attack that affects a given pool, the ZEC supply across all other pools cannot be diluted.

Eventual Supply Integrity

Because this proposal also requires all pools to have a finite duration, Turnstile Enforcement will ensure that the ZEC supply will eventually reach the designed emission schedule even in the face of previous counterfeiting attacks, so long as all newer pools haven't also suffered from a successful counterfeiting attack. This guarantee is called Eventual Supply Integrity.

It is important to clarify for all users that Eventual Supply Integrity and Turnstile Enforcement DO NOT protect individuals or their funds from damage or loss in the event of a counterfeiting attack. These mechanisms also do not recover funds or unroll damage from such an attack, nor do they by themselves guarantee an attack is detected or that an attacker's identity or behavior will be uncovered.

These mechanisms only protect the long term systemic properties of the total ZEC supply which ensures their scarcity into the future which allows the Zcash blockchain to potentially recover fully from counterfeit attacks.

Entrance and Exit Turnstile Mechanics

Every pool is associated with two "turnstiles": the Entrance Turnstile and the Exit Turnstile.

Whenever funds transfer from one source pool to another destination pool, they pass through the Exit Turnstile of the source pool and through the Entrance Turnstile of the destination pool. This must always occur within the scope of a single Zcash transaction with atomic semantics. In order for any transaction making this transfer to be valid it must meet all requirements imposed by any Entrance or Exit Turnstiles (as well as all other protocol requirements for the underlying pools and transaction semantics). If this is not the case, the transaction is invalid.

Entrance Turnstile Mechanics

The Entrance Turnstile intermediates all transfers of funds into the pool from any other pool. Throughout a Pool's Lifecycle, the Entrance Turnstile may be open or shut. When an Entrance Turnstile is open transfers proceed as expected, and the pool's balance is incremented by the amount passing through. When an Entrance Turnstile is shut, all transactions which would transfer funds into the pool from any other pool are prohibited. When this is the case, no funds may enter the pool, and the balance may never increase.

Exit Turnstile Mechanics

The Exit Turnstile intermediates all transfers of funds out of the pool into other pools. When value passes through an Exit Turnstile, the exiting value (before subtracting penalty fees, see below) must be less than or equal to the pool's balance. Otherwise, any block containing the transaction will be rejected due to Turnstile Enforcement (see above).

Security Note: The mere existence of a transaction which would push the balance negative is sufficient evidence that a counterfeiting attack has occurred. However, since the transaction is not valid by consensus rules, there is (currently) no on-chain consensus mechanism for signaling that a counterfeiting attack has been detected. This should be addressed either by network protocol improvements, or by consensus protocol improvements in a future upgrade.

Throughout a Pool's Lifecycle, its Exit Turnstile always has a single well-specified Legacy Pool Penalty Fee Rate, which is a percentage from 0% to 100% inclusive. As a pool approaches the end of its lifecycle, this penalty ramps up from 0% all the way to 100%.

When any transaction includes a transfer of funds out of any Exit Turnstile, it must allocate a proportion of those funds to a Legacy Pool Penalty Fee according to the Penalty Fee Rate of that Exit Turnstile.

Legacy Pool Penalty Fees must be contributed into existing transaction fees within the same transaction in which the originating funds passed through the associated Exit Turnstile.

BUG / FIXME: Assigning Penalty Fees to transaction fees allows miners to exit older pools with no fee, since they can pay themselves this fee in their own blocks. This design choice is retained in this proposal to limit the proposals complexity. However, if future protocol upgrades introduce different mechanics that would make a good candidate destination for these funds (such changes to transaction fee mechanics, or "privacy incentivization funds"), those future proposals should consider altering the destination of these Penalty Fees.

Rationale: Legacy Pool Penalty Fees serve three functional purposes:

1. They incentivize users to migrate out of older pools before penalties apply or grow more severe,
2. They remunerate block producers as compensation for operating consensus validation logic for older technology (which has attendant operational costs and risks), and
3. They maintain the overall ZEC supply which we posit makes ZEC valuation simpler for more users (in contrast to an alternative of "burning" the funds, which would reduce the ZEC supply).

Lifecycles

Every pool proceeds through a sequence of Lifecycle Stages along a Lifecycle Timeline. The stages define common / consistent behavior for users across pools, allowing the userbase to migrate in an orderly fashion from older technology to newer technology over time.

Lifecycle Stages

In unexceptional cases, pools proceed through a sequence of stages:

1. Active
2. Closing
3. Penalized
4. Removed

Additionally, in certain kinds of emergencies, a pool may be placed in the frozen (emergency) state by an explicit protocol upgrade.

FIXME: Incomplete below.

---

TODO: Define transition timelines and mechanics.

TODO: Define penalty fee ramp up.

TODO: Define pool removal.

TODO: Explain some alternative lifecycle states we've considered; share Str4d's helpful table.

TODO: Re-examine what this means for T-addr pool, which I haven't thought much about yet.

TODO: Get lots of wide-net feedback early.

TODO: Read the entire forum thread on pool retirement.

TODO: Define supported/unsupported use cases, motivation, rationales better.

TODO: Gather and link to alternative proposals.

Use Cases

Supported Use Cases

xxx fixme

Precluded Use Cases

This proposal precludes use cases by dint of assuming all pools have a finite lifetime and stipulating all pools are constrained to Turnstile Entrace/Exit accounting.

• Unbounded cold storage is not possible.
• Delivering sufficiently old transactions to the network is not possible.
• Remaining in a single shielded privacy set indefinitely (without passing through a turnstile) is not possible.

[daira]

I disagree with conflating the goals of maintaining supply integrity and reducing cryptographic risk, with the mechanism of partitioning note traceability sets by pool and requiring value-revealing transfers between them.

Both goals are potentially possible with different mechanisms, such as (but not limited to) the one described in https://github.com/zcash/zcash/issues/2373 .

[pacu]

: Assigning Penalty Fees to transaction fees allows miners to exit older pools with no fee, since they can pay themselves this fee in their own blocks.

Miners could hold a substantial part of the ZEC supply. Them being able to avoid penalties, could pose as an incentive to delay the adoption of newer pools. In that case, we could think of only allowing miner's rewards to be sent to best pool addresses at all times.

[nighthawk24]

In that case, we could think of only allowing miner's rewards to be sent to best pool addresses at all times.

Agree, the newest mining client release should only allow sending of block rewards to the latest shielded pool.

[nathan-at-least]

Here's a table summarizing an ECC engineering discussion that @str4d put together. This is a useful reference to triangulate some known implementation possibilities for pool states.

Shielded Pool State	Needs	Into?	Within?	Out?	Turnstile type & what is revealed	???
Latest / Active	⚙️🛡️	✔️	✔️	✔️	Two-way, tx value
Closing	⚙️🛡️	❌	✔️	✔️	One-way, tx value	🤔
Exit-only	⚙️🛡️	❌	❌	✔️	One-way, tx value
Redeem-only	⚙️	❌	❌	✔️	One-way, note values	🤔👀
Locked (Emergency)		❌	❌	❌	None, Temporary	⚠️
Removed		❌	❌	❌	None, Permanent	☢️

⚙️ = Requires consensus-level support 🛡️ = Requires a circuit 🤔 = Assumed to be confusing to users (Nate’s intuition, not verified) 👀 = No privacy protections for individually identifiable spent notes. (Note: every option reveals a transaction’s aggregate in/out amounts as part of the “turnstile design policy”.) ⚠️ = User Funds are inaccessible. ☢️ = User Funds are Permanently Destroyed!

(Note: I cut'n'pasted from a google doc and the result looks like a good replica, but there may be bugs I missed.)

[nathan-at-least]

From https://github.com/zcash/zips/issues/452#issuecomment-795594957 :

: Assigning Penalty Fees to transaction fees allows miners to exit older pools with no fee, since they can pay themselves this fee in their own blocks.

Miners could hold a substantial part of the ZEC supply. Them being able to avoid penalties, could pose as an incentive to delay the adoption of newer pools. In that case, we could think of only allowing miner's rewards to be sent to best pool addresses at all times.

Great suggestion! I was imagining future separate changes to things like transaction fees would help us here, but this approach is much simpler and can be integrated into the proposal as-is.

Edit: I realized this would still allow a miner to store funds in a Penalized Pool without penalty. They can migrate whenever they want prior to the pool being Removed so long as they can mine a block where they include their own migration txn paying themselves the penalty fee. I still think it's an improvement simply by requiring miners to use the latest pool which hopefully will be sticky for some portion of them.

[nathan-at-least]

From https://github.com/zcash/zips/issues/452#issuecomment-795236358 :

I disagree with conflating the goals of maintaining supply integrity and reducing cryptographic risk, with the mechanism of partitioning note traceability sets by pool and requiring value-revealing transfers between them.

Both goals are potentially possible with different mechanisms, such as (but not limited to) the one described in zcash/zcash#2373 .

This seems like a great candidate for a different proposal to pool lifecycles, because in Unlikely Coast this is probably a baked in assumption that the two goals are addressed by turnstiles. (TODO: spell out that assumption explicitly.)

One thought if an improvement over turnstiles becomes feasible in the future but not the short term, we could adopt something like Unlikely Coast for the time being, but switch to a new Lifecycle model when a turnstile replacement ships. That could alter the fate of any pools at that time, or it could be introduced for new pools where old pools are grandfathered in.