fix: reject trailing bytes when decoding transactions

[fgimenez]

The RLP decoding of transactions in EngineAPI is not strict, it allows excess bytes to be appended to a transaction and still decode successfully. According to the Yellow Paper, the transactions must be well-formed RLP, with no additional trailing bytes.

With these changes we check in decode_enveloped if there are any remaining bytes in the input data after decoding, failing with RlpError::UnexpectedLength if so.

[Rjected]

looks ok to me, however i'd like someone else to review this as well in case i am missign context on why we did not perform this check, maybe @Rjected or @mattsse since the likely cause is networking

yes, networking actually requires trailing bytes sometimes because you may be decoding more than one transaction in a list for example. In rpc (including engine) you do not want trailing bytes. This was probably just an oversight

Is this change also doable in alloy?

yes, but only for the enveloped encoding, reasoning above