Research transaction fees

[takemiyamakoto]

I think we can use triggers for this (e.g., post-execute triggers).

[e-ivkov]

In the Trigger RFC we discussed why using triggers for transaction fees might be difficult. And decided to keep it either as a WASM Plugin or just highly configurable at runtime system.

[s8sato]

Requirements for the transaction fees I understand at the moment:

• Fees should be determined before execution
• #1214 as atomic weights that will be summed to an executable weight
• Estimate complexity of wasm executables. How are other blockchains using wasm as a smartcontract language calculating fees?
• Define a rate between the weight and the "base currency" in the chain

Do you have any updates about this @takemiyamakoto ?

[mversic]

some additional resoures

https://soramitsu.atlassian.net/wiki/spaces/~61e107308534980073335d24/pages/5080711170/Iroha+2+MVP+Fees+structure+for+CBDC

https://build.avax.network/docs/api-reference/guides/txn-fees

[s8sato]

Gas consumption is measured across four dimensions:

1. Bandwidth The transaction size in bytes.
2. Reads The number of state/database reads.
3. Writes The number of state/database writes.
4. Compute The compute time in microseconds.

The total gas consumed ($G$) by a transaction is:

$$G=B+1000R+1000W+4C$$

This provides useful insights. To estimate each dimension:

• Bandwidth: The client can calculate the transaction size by itself.
• Compute: The only way I can think of right now is statistical prediction based on other dimensions.
• Reads and Writes: I have the following idea:

Currently, user logic can contain queries and instructions in any arrangement. As an alternative, we could require that queries be placed at the beginning and instructions at the end:

fn read_request() -> ReadRequest; // static
fn write_request(input: ReadResult) -> WriteRequest; // dynamic, return value depends on the state

Note that these functions do not execute queries or instructions directly; they merely return requests to the host.

Write requests are state-dependent but should be within a predictable range. We can allow the logic author to declare the maximum possible set of write requests:

impl From<WriteRequest> for WriteRequestAbstract {}
impl PartialOrd for WriteRequestAbstract {}
fn write_union() -> WriteRequestAbstract; // static

The host would then validate the condition actual_write_request <= expected_write_union at runtime.

[s8sato]

Regarding wasm executables, fees could simply be proportional to the maximum allowed complexity -- the fuel parameter. What do you think?

[0x009922]

Regarding wasm executables, fees could simply be proportional to the maximum allowed complexity -- the fuel parameter. What do you think?

@s8sato it is simple, but might be excessively large for very simple smartcontracts.

I have an idea:

• Compute fee based on space transaction occupies (includes metadata, instructions count, wasm blob size) and, if it is Executable::Wasm, also base fee on the fuel spent on the WASM
• Transaction authority must specify how much fuel they are ready to spend. Executor than could check before WASM execution that the account actually has enough to pay that maximum price. Then Iroha executes WASM and tells to executor how much fuel it took. Then executor deducts actual fee from the account.
• Should be there something different for triggers? They are also just triggering Executable, and same rules apply (not sure about the space occupation in this case).

Pros:

• Takes into account both space and time complexity into fee calculation
• No need to think about "precomputing" fees

Concerns:

• Executable::Instructions and Executable::Wasm are very different by nature. I expect it to be nearly impossible to balance "fair" fee calculation for them. Probably, we will end up with a situation that it is either always less expensive to use instructions, or always less expensive to use WASM-based approach. Is it okay?
• We store both successful and failing transactions in the blockchain. Failing transactions still take both space and compute. The only difference is that their effect is not committed.
• Making queries might be also very complex, and it is done outside of the WASM. Therefore, it is not included in the fuel WASM takes.

[mversic]

• Compute fee based on space transaction occupies (includes metadata, instructions count, wasm blob size) and, if it is Executable::Wasm, also base fee on the fuel spent on the WASM

In the case of wasm it should sum up both fuel and the price for every instruction because wasm ends up calling ISI on the wsv as do non-wasm transactions

[mversic]

Compute fee based on space transaction occupies (includes metadata, instructions count, wasm blob size) and, if it is Executable::Wasm, also base fee on the fuel spent on the WASM

this sounds to me as if you're proposing to have 2 different types of fees, i.e. space fee and compute fee. While we can have space fee I think that we must have a compute fee for every ISI. Then the compute fee of a wasm is the sum of every ISI compute fee it executes and it's own fuel consumption

[0x009922]

@mversic Yes, I think I agree with you. It sounds reasonable to "bill" every executed ISI.

this sounds to me as if you're proposing to have 2 different types of fees

Yes, but not exactly. I don't think it is needed to have multiple different fees. I am speaking about the different measures we can take into account to compute the final, single, composite fee.

To summarize my idea (not a final proposal): we extend Iroha core so that it provides the following to the executor:

• Measures
  • Overall size of transaction. Or maybe only its metadata
  • The fuel some smartcontract consumed
  • Computational complexity of each query executed?
• Control
  • Post-execute hook: ability for executor to make final decision/extra actions after the transaction is executed. Here is when it computes the final fee and ensures the deduction is possible and no rules violated. If so, it could deny the transaction.
  • Maybe, an ability for executor to store some arbitrary "context" data throughout the hooks executed during a single transaction. So, for example, it could create some context at the beginning of a transaction. Then, for each visit_instruction (and visit_query) it could store some billing info in that context. Then, in the post-execute hook, it computes the final billing based on the data in that context. This context could be just plain Metadata, or whatever binary data.

[0x009922]

@aoyako: Fee processing

Fee Processing Proposal The current implementation with a custom executor has certain limitations on what can be stored and modified in the core. Since we want to keep fee information within transactions while ensuring the executor remains responsible only for individual ISI processing, the following approach seems optimal.

Proposed Solution

1. The initial fee amounts are included in the instruction (possibly as a wrapper around existing structures).
2. Fee validation:
   • [simple] For simple ISI, the fee is validated, and the amount is transferred to the treasury.
   • [was] When executing a smart contract, the WASM runs with limitations (fuel, memory) based on the initial fees instead of network defaults.
3. The submitted unwrapped ISI is processed the same way as before.
4. [wasm] After WASM execution, an event is emitted containing the allocated and consumed resources.
5. [wasm] A trigger fires on the event (step 4) to refund certain amounts from the initial fees.

Summary of Execution Flow Simple ISI: Set initial fees → Validate fees → Transfer fees → Execute Smart Contract (WASM): Set initial fees → Transfer fees → Execute with limits → Emit execution event → Refund based on usage

Since steps involving triggers require changes to the core, I would appreciate any feedback on this proposed pipeline.

@takemiyamakoto It's better to set up a transaction pipeline for processing transactions and include distinct phases. In Polkadot it works this way: pre-execute → execute → post execute. For fees, fee checks are done in pre-execute phase and can include custom code, but fees are paid at the end. Pre phase just makes sure there is enough to pay the fee before executing. This allow flexible solutions, like we made xorless fees in Polkaswap where you can pay in other tokens, but they are swapped to xor in the post-execute phase (amounts are quoted and checked in the pre-phase to make sure there is enough money and liquidity). We also allow you to swap to XOR on Polkaswap without having any XOR, as long as you swap at least enough to pay fees, as the amount of XOR to pay fees will be taken out at the end after the swap.

By having pre- and post- execute hooks that are set by customization code, you can make a flexible and robust pipeline for transaction processing that can cover all the possible use cases.

[0x009922]

Since we want to keep fee information within transactions while ensuring the executor remains responsible only for individual ISI processing

1. Why is it important to keep within transactions?
2. Is it only responsible for individual ISI processing? AFAIK it has visit_transaction entrypoint.

In Polkadot it works this way: pre-execute → execute → post execute

This sounds quite simple and flexible. I wonder what prevents us from implementing it in a similar way?

[aoyako]

1. Why is it important to keep within transactions?

Initially, I thought it should be kept in a single transaction to ensure that account assets don't change. It is possible to distribute it in multiple transactions, but some rules to match instruction/transaction and fee validation/payment records should be implemented.

3. Is it only responsible for individual ISI processing? AFAIK it has visit_transaction entrypoint.

That is true, and transactions are also possible.

This sounds quite simple and flexible. I wonder what prevents us from implementing it in a similar way?

I think this is a good default pipeline. As for pre- and pos- execution, are there requirements where they should be implemented (core, or wasm)? Previous suggestion https://github.com/hyperledger-iroha/iroha/issues/1177#issuecomment-2731671290 can be extended to include those phases in the custom executor.

[0x009922]

Initially, I thought it should be kept in a single transaction to ensure that account assets don't change. It is possible to distribute it in multiple transactions, but some rules to match instruction/transaction and fee validation/payment records should be implemented.

Ah, I see. Well, I think it makes perfect sense for fees to be a part of a single transaction with the main operation. It makes sense for them to be a single, atomic, nuclear operation.

As for pre- and pos- execution, are there requirements where they should be implemented (core, or wasm)?

IMO, the mechanism of pre- and post- hooks is generic and useful enough to be a part of the core. I mean, make these hooks happen always for any executor, and it is up to the executor what to do with them. Does this answer your question?

[aoyako]

One possible approach designed with @0x009922

Pipeline

1. User calls InitTransaction (custom isi or trigger) with payload (::InitTransaction) {transaction_id, instruction}
   • Execute the instruction or wasm without committing it
   • Calculate total fee of the operation (based on account config, fixed/percent, default values, fuel etc.)
   • In trigger (or account's) metadata, create an entry for the transaction (with transaction_id) with the total fee
   • Emit custom event InitTransactionReturn {transaction_id, fee}
2. User listens for events InitTransactionReturn {transaction_id, fee}
3. User accepts/rejects fees
4. User calls CommitTransaction for {transaction_id}
   • The fee amounts are subtracted from the account
   • The transaction is committed

[0x009922]

My proposed approach is addressing CBDC needs specifically, and it seems like it pretty much solves their problem without us needing to introduce very complicated infrastructure and execution model.

Requirements

In essence, what CBDC needs:

• Composite fee - with fixed ("fixed absolute amount of fee paid with every transaction") and variable ("flexible relative amount of fee paid with every transaction, which is calculated as N percents of the transaction amount") components
• Admin account that configures fixed and variable fees
• Admin account should be able to create accounts with default fees config.
• Admin account should be able to customise fees config per-account later.
• They have 2 certain business operations performed by the clients (i.e. accounts registered by the admin account):
  1. "Instant Money Transfers"
  2. "Flexible Withdrawals"
• Client, while performing any of these certain business operations, should be able to swiftly receive fee calculation "on the fly", while inputting each number into some input. And then, if they confirm, the operation will be actually performed.

I think these requirements could be addressed with the current infrastructure (using triggers) + custom events (which are relatively easy to implement). With custom events, it becomes possible to implement a sort of on-chain RPC, client-server interaction: clients execute the trigger with a certain payload, trigger performs changes/stores state in some metadata, and emits a custom event as a response to the client.

Actually, it isn't even needed to have custom events - trigger could store its response in some metadata as well. It is just not as pretty.

Trigger RPC

On high-level, the API of the trigger might look like this:

enum TriggerOp {
  // Admin ops

  /// Set global defaults
  SetFeesConfig(FeesConfig),
  /// Register an account.
  CreateAccount {
    name: Name,
  },
  /// Set for an account
  SetAccountFeesConfig {
    account: AccountId,
    config: FeesConfig 
  },

  // Client ops

  /// "Dry run" of the operation - nothing is done,
  /// trigger only replies with the calculated fee
  InstantMoneyTransferDryRun {
    id: RequestId,
    // whatever are the options of the instant money transfer
    // - amounts, target etc
  },
  /// Initiation of the actual transaction,
  /// trigger will apply all needed business rules and calculations,
  /// withhold the fee, and "fix" this transaction
  InstantMoneyTransferBegin {
    id: RequestId,
    // opts
  },
  /// And if the client approves the final actual fee, it sends this
  /// "commit" message with the received transaction id
  InstantMoneyTransferCommit {
    tx_id: String,
  },
  /// In case if client doesn't want to proceed, but wants to return
  /// the withheld fee
  InstantMoneyTransferRollback {
    tx_id: String
  }

  // same set of opts for "Flexible Withdrawals" whatever that means
}

enum TriggerOpResponse {
  InstantMoneyTransferDryRun {
    id: RequestId,
	// the calculated fee for this particular request
    fee: Numeric
  },
  InstantMoneyTransferBegin {
    id: RequestId,
    fee: Numeric,
    tx_id: String
  }
}

type RequestId = String;

struct FeesConfig {
  fixed: Numeric,
  percent: Numeric
}

Trigger implementation

Internally, what the trigger does:

• SetFeesConfig - sets global defaults somewhere in its own metadata? Or in any metadata, up to the implementor.
• CreateAccount - registers an account with appropriate permissions and metadata.
• SetAccountsFeesConfig - updates account's metadata with the specified config
• InstantMoneyTransferDryRun/InstantMoneyTransferBegin: based on business rules and accounts/global fees config, calculates the fee. Checks whether the client has sufficient balance. If it is DryRun, just returns the fee and whether it is enough. If it is Begin, it "withholds" the fee from the account (on some service account?) and somewhere stores the information about the "transaction" - its random id, fee, options.
• InstantMoneyTransferCommit - trigger looks up the transaction by id and, if everything is ok, applies it.
• InstantMoneyTransferRollback - trigger cancels the operation by refunding the withheld fee and updating metadata.

Notes

Concerns:

• Registered accounts must not be permitted to perform side effects of the trigger on their behalf. I.e. they should only be permitted to do any transactions via this trigger.
• Cleanup of temporary data. For example, clients could begin and not finish transactions. Could a simple time-trigger cleanup stale transactions?

Interesting outcomes:

• All requests to the trigger are transactions, stored on the chain. The payload is openly visible. Will be even better after #4968
• Trigger mutates external state, and it is actually visible in the blockchain (at the moment)

[s8sato]

Here are my comments at the moment, though I'm still exploring major public chains:

• We need a mechanism to ensure that what is validated during pre-execution remains valid long enough, such as "locking" account balances at the pre-execution phase. Otherwise, an account may lose solvency due to interruptions from other transactions.
• Just an impression, but the idea of reserving and approving transactions reminds me of multisig. It seems like converting every transaction into a multisig transaction between the admin and the client. Their business logic might be shareable to some extent.

[0x009922]

We need a mechanism to ensure that what is validated during pre-execution remains valid long enough, such as "locking" account balances at the pre-execution phase.

Yes, that should totally be enforced. I think it is within the power of triggers/executor, and is up to the implementation to make this mechanism robust. (in other words, there is no need to introduce any changes to the core)

[s8sato]

• They have 2 certain business operations performed by the clients (i.e. accounts registered by the admin account):
  1. "Instant Money Transfers"
  2. "Flexible Withdrawals"

The CBDC use case appears to involve a transaction that consists of a single built-in instruction (Transfer).

Does this mean we can skip the complex part of fee estimation found in most public blockchains -- specifically, associating the fee with computation cost?

[aoyako]

The CBDC case has been delayed for some time. Meanwhile, a more fundamental approach can be implemented to cover all potential cases (including wasms). As for the network fees, from @takemiyamakoto

For ISIs, you should just have a flat fee I think the best approach is the Solana/Ethereum approach

For business fees, both fixed and percentage fees can be used.

[s8sato]

Imo, the key differences lie in UX and how each addresses the halting problem:

• Solana: The transaction either succeeds by taking the entire user-specified fee, or fails if the fee is insufficient.
• Ethereum: Automatically optimizes the fee through repeated pre-executions using a binary search approach.

[s8sato]

Fee deduction and refund logic

To ensure the logic is MECE, we can define the decision flow as follows, for example:

flowchart TD
    S((Start))
    A{Is the account balance sufficient for the fee deposit?}
    B{Is permission validation successful?}
    C{Is the step deduction and execution successful?}
    D{Are there remaining execution steps?}
    O(Conclusion: No operation)
    P(Conclusion: Full deduction)
    Q(Conclusion: Payment = Deposit – Refund)
    X>Commit as Rejected]
    Y>Commit as Approved]

    S --> A
    A -- No --> O
    A -- Yes --> B
    B -- No --> O
    B -- Yes --> C
    C -- Invariant Violation --> O
    C -- Deposit Shortage --> P
    C -- Yes --> D
    D -- No --> Q
    D -- Yes --> C
    O --> X
    P --> X
    Q --> Y

[s8sato]

Updated the decision flow to reflect permission validation and the execution-and-deduction cycle

[aoyako]

On second thought, there is no need to set individual fuel limits for the executables. They can be executed as usual, but later, checks can be added to verify that the spent fuel does not exceed the reserved amount. With this approach, users can also see the fuel required to successfully run the executable (even in the case of a failed transaction) and reserve accordingly for the next call.

[s8sato]

@aoyako

• Unless the user specifies a fuel limit, the system must impose one during pre-execution. If the limit is too high, it permits resource waste; if it is too low, it may reject valid (but complex) user logic.
• Indeed, when it comes to fuel as part of the overall cost, we do not need a procedure like Ethereum's binary search, since the exact consumption can be retrieved via wasmtime::Store::get_fuel().

@s8sato

Defect. Users can launch the following DoS attacks with no risk:

• Declaring fees beyond their payment capacity
• Attempting unauthorized transactions
• Attempting transactions that violate invariants