Fee estimation endpoint?

[ngua]

Describe your idea, in simple words.

• A new endpoint, perhaps named EvaluateTxFee, could be added to estimate transaction fees
• This would take a serialized transaction and return an integer representing tx fees in lovelace. Depending on the libraries used to implement this, we'd also need to pass the expected witness count with the request

Unresolved questions:

• Which protocol would this live under? Perhaps local tx submission would be the best as it wouldn't be a state query
• An alternative: extend the EvaluateTx endpoint to include a fee estimate?

Why is it a good idea?

• This would be a good counterpart to the EvaluateTx endpoint
  • It's often necessary to calculate fees and ex units in tandem as they affect each other

Are you willing to work on it yourself?

• For the sever, yes. Either I or someone from the CTL team could work on that part. We already have a custom fee estimation endpoint written using Cardano.Api.Shelley. Perhaps we could adapt that and integrate it into Ogmios instead
• I'm personally not knowledgeable enough about the clients to handle that part, so I would likely need some assistance with that

[KtorZ]

Sounds like a reasonable idea to me. Remains to know what would be the most convenient interface for that.

Doing proper fee estimation is not straightforward. I actually ran not later than yesterday into a bug where cardano-cli would under-estimate fee on a simple transaction :(

Ideally, I imagine that this could work from a raw transaction, with fees set to 0 and providing an estimation based off that. It'd leverage the network to resolve UTxO and figure out the type of witness needed.

[ngua]

Great! I'm happy to hear you're open to the idea and thank you for responding so quickly

In terms of implementation, I'm not sure if the following would be appropriate, but we've used Cardano.Api.evaluateTransactionFee to calculate the fee using a CBOR-encoded tx and a key witness count. Although unfortunately this alone isn't sufficient and we've had to include some "fee finalization". For our backend Haskell service, this currently looks like:

finalizeTxFee :: Integer -> AppM Integer
finalizeTxFee fee
  -- `integerLog2` would fail on zero,
  -- since logarithm of zero is mathematically undefined
  | fee == 0 = pure 0
  | otherwise =
    asks protocolParams <&> \pparams ->
      let feePerByte =
            fromIntegral (Shelley.protocolParamTxFeePerByte pparams)
          -- the required number of bytes is calculated twice to
          -- be able to handle a possible integer overflow
          feeBytes =
            bytesNeeded (fee + bytesNeeded fee * feePerByte)
       in fee + feeBytes * feePerByte
  where
    -- Calculates the number of bytes that a given integer will take
    -- when CBOR encoded.
    bytesNeeded :: Integer -> Integer
    bytesNeeded n =
      let predicate = (>= succ (Math.integerLog2 n `div` 8))
       in fromIntegral . fromJust $ -- using `fromJust` here is safe
            List.find predicate [2 ^ x | x <- [(0 :: Int) ..]]

This approach was discovered by @errfrom after months of persistently insufficient fee estimates. At any rate, something similar could serve as a starting point or inspiration for a future implementation

I agree that the interface needs some careful thought, and we would also need to clearly document some of the major caveats we've found so far: the script integrity hash must be set in the tx body and all redeemer/datum witnesses must be attached (along with a current estimate of ex units for each redeemer!)

[KtorZ]

Yes, I had to write some very ugly "temporary work-around" in cardano-wallet back in the days for Plutus-Script-locked inputs; mostly because of the transaction integrity hash that is added to the body and the redeemer units (making the size, and thus fee bigger)... those can only be added at the very end of the process, which makes the fee calculation a kind of mutually recursive setup where there's actually no guarantee to ever converge :upside_down_face: (if the underlying script relies on say, value in the change output, it may be impossible to find the right fees)..

Rather than estimating fee, I would rather provide a way to calculate the min-fee based on a pre-constructed transaction; and give maybe hints as for how to pre-construct a transaction that would result in a stable fee once fully completed.

[klntsky]

We are going to switch to CSL for fee calculation.

https://github.com/Plutonomicon/cardano-transaction-lib/pull/866