gear icon indicating copy to clipboard operation
gear copied to clipboard

Handle code instrumentation cases in `process_queue`

Open techraed opened this issue 2 years ago • 2 comments

Problem to Solve

Need to resolve these todos https://github.com/gear-tech/gear/blob/ab6c1d66195aafa29c9840cc6f4e0c36db536b54/pallets/gear/src/lib.rs#L751-L772

The problem is quite serious. We should not stuck in the state, when most of codes can't be re-instrumented. To avoid that:

  • [x] Research reasons for instrumentation failures (https://github.com/paritytech/wasm-instrument)
  • [ ] Depending on the results of the research, provide such a re-instrumentation handling, that it won't block normal working of process_queue, i.e., actor model scenarios will go on, messages will be sent/received/processed

techraed avatar May 04 '22 04:05 techraed

After some research of wasm-instrument and parity-wasm crates we decided to just call except on the result of the re-instrumentation. Some reasons for that:

  1. Serialization/deserialization errors can't occur on successfully deserialized/serialized module that is going to be re-instrumented. As long as wasm module stays valid, we are safe from these errors.
  2. Re-instrumentation doesn't affect wasm module except for inserting new weights for instructions, so it stays valid as it was.
  3. There are only 2 possible ways of re-instrumentation failure, that can actually be controlled by us. The first and the easiest to control one is a deletion of the existing instruction (it's weight) from the instrumentation Rules. We can forbid deleting any of instruction's weight using Rust's strong types. For instance we can perform explicit exhaustive matching of the enum with instructions without using any _ pattern, so guaranteeing that for every instruction the weight is defined.
  4. The other one is a possible overflow while calculating sum gas cost of the wasm block. Each instruction cost is a u32 type value. If we define too large weights for instructions, we can end up re-instrumentation with an error (wasm-instrument uses checked_add operation and returns an error if the result of the operation is None). Let's check whether overflow is possible in current weights.
  5. Currently, the most expensive instruction has cost of 1_713_000 / 100. Gear program's max size is 512 * 1024 byte. Imagine, that each instruction weighs 4 bytes. Also imagine, the weight of the all wasm program is actually the weight of the one block full of instructions. So maximum possible amount of instruction equals to 512*1024 / 4 = 131_072. So the maximum cost of the block is 131_072 * 1_713_000 / 100 = 2245263360 < u32::MAX.

techraed avatar Aug 16 '22 15:08 techraed

Real example, how the size can be less than 512 kiB, but instrumentation fails due to total gas overflow: https://github.com/gear-tech/gear/issues/1353

techraed avatar Aug 23 '22 08:08 techraed