Review usage of `volatile` with `Interlocked`

[bart-vmware]

Steeltoe uses volatile fields and Interlocked.* methods. And sometimes together, which isn't how it is supposed to be used. See https://stackoverflow.com/questions/1186515/interlocked-and-volatile.

[bart-vmware]

The most-official and most-up-to-date information I could find is at https://github.com/VSadov/runtime/blob/main/docs/design/specs/Memory-model.md, originating from https://github.com/dotnet/runtime/issues/63474. Also good info at: http://igoro.com/archive/volatile-keyword-in-c-memory-model-explained/.

My conclusions from that are:

• volatile fields are always atomically read/assigned (no tearing).
  • A long field can't be declared as volatile (it would tear). Volatile.Read/Write provides non-tearing.
• Reading/writing a volatile field flushes all CPU caches. So reading gives a fresh snapshot.
  • If a field is not declared as volatile, Volatile.Read gives you the same freshness (assuming non-cached writes).
• Accessing a volatile field limits the potential for reordering instructions (fences).
• volatile fields result in a full cache refresh (so that nested object properties are consistent).
• Interlocked.* methods provide at least the same guarantees as volatile fields.
• Interlocked.CompareExchange enables detection of concurrent changes (conditional update), whereas when using volatile, the latest writer always wins (potentially undoing the work from others).
• If a field is being changed from multiple sources (with potentially conflicting updates), use a lock to synchronize. Atomic replacement of the field reference would effectively undo concurrent changes.