Memory Gas Costs

[Stebalien]

We need to know:

• Cost to randomly read one word.
• Cost to randomly write one word.
• Write cost per byte (bulk).
• Read cost per byte (bulk).

From this, we can derive most of the "variable" instruction costs in the FVM.

(where cost is time)

From that, we can use the baseline 10gas/ns to calculate the correct gas values.

To do this, we likely need to allocate a large memory (e.g., 1GiB) and read-write random memory locations. That way, we really measure memory latency.

However, the per-byte cost should be for sequential access (e.g., read a range from some random offset).

[Stebalien]

Measurement strategy....

1. Perform setup.
2. Log nothing 10 times.
3. Perform the test (ideally many times).
4. Log once.

We can use the first 10 logs to determine accurate syscall overhead, and the time between the last two logs (minus that overhead) will be the time spent running the test itself.

Read Latency

To time reads, we need to:

Setup:

Allocate 256MiB of memory, initialize to random addresses in the range [0, 256MiB).

Test:

1. Read the first address.
2. Repeatedly dereference addresses (some fixed number of times). Prefer to use a long sequence of instructions, not a loop.

Memcpy

Once we have the read timing, we can measure memcpys. We'll need to re-run this test for different copy sizes (N), likely powers of two (fuzzed a bit to break alignment).

Setup:

Allocate 256MiB+N of memory (or a bit more, initialize to random addresses in the range [0, 256MiB).

The +N avoids an out of bounds error if we copy past the end.

Test:

Like above, we'll be reading addresses from memory. However, instead of just chaining reads, we'll need to:

1. Starting at address 0.
2. Read three addresses from memory (sequential loads).
3. Memcpy (using the bulk memory instruction) some fixed number of bytes N from the first address to the second.
4. Go back to step 2 starting at the third address.

When determining gas values, subtract gas for one random memory access for each iteration.

Fill

Finally, we need to measure the cost of "filling". Again, we'll need to repeat for various sizes N.

Setup:

Allocate 256MiB+N of memory, initialize the first half to random addresses in the range [0, 128MiB).

Test:

1. Starting at address 0.
2. Read an address (A).
3. Multiply by 2 (B).
4. Fill N bytes at address B with some X (chosen when writing the benchmark).
5. Go back to 1 but read the address from memory index A instead of 0.

[Stebalien]

For now, I'm going to assume:

1. 0.5gas per memory byte copied and/or filled.
2. 100gas per random read (10ns which is about the random access speed of most DDR4 RAM).

[Stebalien]

Look into the affects of non-aligned memcpy and aligned memcpy.

[Stebalien]

We've benchmarked this in the EVM (#1281) and have determined that it won't be an issue in M2.1. So we're punting to M2.2

[Stebalien]

Specifically, we're looking to correctly set the memory_access_cost gas fee: https://github.com/filecoin-project/ref-fvm/blob/280d80503f950a1934e3d60910d659fad685f9c7/fvm/src/gas/price_list.rs#L517. We currently don't charge any additional gas here but we likely will need to do so if/when we support user-deployed wasm actors.