Cache frequently accessed structures in `PartitionStore`

[tillrohrmann]

To avoid the serialization cost when accessing data stored in the PartitionStore, we could add an LRU cache for frequently accessed data. Prime candidates could be the deduplication table and the InvocationStatus.

[slinkydeveloper]

Could this LRU cache be completely hidden behind the InvocationStatusTable impl? In particular once we get rid of these https://github.com/restatedev/restate/issues/276

[tillrohrmann]

I've ran some simple experiments and caching of the InvocationStatus didn't have a measurable impact on throughput.

[slinkydeveloper]

I've ran some simple experiments and caching of the InvocationStatus didn't have a measurable impact on throughput.

Did you by any chance tried with the neo invocation status to see if the new protobuf layout has any either positive or negative impact?

[tillrohrmann]

Did you by any chance tried with the neo invocation status to see if the new protobuf layout has any either positive or negative impact?

No, I only tried it with the old InvocationStatus. Given that reading the InvocationStatus took ~1.5% of time in the flamegraphs, I wouldn't expect a super noticeable change. This doesn't mean that the we shouldn't optimize these things, though.