Communicate intention for weak references when scanning objects

[wks]

TL;DR: The Scanning::scan_object API should have an additional argument to tell the VM binding MMTk-core's intended way for handling weak references.

• This is not strictly necessary (although nice to have) for porting the OpenJDK-specific Reference processing code from the lxr branch to master. We can implement OpenJDK-style reference processing in mmtk-openjdk without modifying mmtk-core.
• This is not necessary for implementing the SATB barrier for ConcurrentImmix, either.
• It will be necessary for LXR or RC-based plans.

What does the lxr branch do?

The lxr mimics the way OpenJDK processes weak references. It gradually discovers Reference objects when computing the (strong) transitive closure, and process discovered references afterwards. The discovery is piggy-backed on the object-scanning code, which is Scanning::scan_object in our API. During reference discovery, the referent field of a weak Reference object is ignored, but the Reference is added to a discovered list.

But at the same time, the write barrier function object_probable_write_slow visits both strong and weak fields. When RC promotes an object, it also visits both strong and weak fields.

Such a requirement needs to control whether Scanning::scan_object should visit weak fields and whether it should do "reference discovery".

Conflicts with the goal of being VM-neutral

MMTk core is designed to be VM-neutral. It needs to support VMs that process weak references in different ways. Our Porting Guide mentioned that there are two basic ways to identify weak references:

1. Registering them when they are constructed. (e.g. JikesRVM)
2. Discover them while computing transitive closure. (e.g. OpenJDK)

So I deem the flags RefScanPolicy::Follow and RefScanPolicy::Discover inappropriate for mmtk-core APIs because "reference discovery" is a concept that OpenJDK uses, but not other JVMs (such as JikesRVM).

Just tell the VM what mmtk-core wants to do

A proper API should communicate the intention of the invocation of Scanning::scan_object. Currently in the master branch, mmtk-core calls Scanning::scan_object for several purposes:

• Computing the transitive closure of strong references (including following further strong references after retaining softly reachable objects and/or finalizable objects). This is for most plans.
• Updating references before compaction. This is for MarkCompact and Compressor.

We can simply encode those two intentions with the following enum:

enum RefScanPolicy {
    StrongClosure,
    RefUpdate,
}

• StrongClosure: Scanning::scan_object should only visit strong fields. Depending on the VM implementation, the VM may add the reference to "discovered lists" or ignore them if they are already registered.
• RefUpdate: Scanning::scan_object should simply visit all fields.

The key is that we don't mention "discover" in the API. The VM binding decide whether to "discover" in the StrongClosure case.

Other potential intentions

RefScanPolicy may take other values if we have other use cases.

• SatbBarrier: We are scanning an object in the slow path of an SATB barrier. It should only visit strong fields because the snapshot should only contain strongly-reachable objects. There are weak-field-loading barriers that handle weak fields by keeping the referent alive.
• RCBarrier: (Correct me if I am wrong) Coalescing RC barrier. It should count both strong references and weak references because we can't handle weak references when an object suddenly becomes not strongly reachable during RC. We may have to conservatively let weak references contribute reference counts, too.
• StrongOnly: Force visiting strong fields, only. This implies there is no need to do any special handling for weak fields or Java-style Reference objects.
• All: Force visiting all fields.

But since barriers are executed in mutators, we need introduce another API function in Scanning. See https://github.com/mmtk/mmtk-core/issues/1375 for more details.

[qinsoon]

There are a few things that I dont understand about the issue

In TLDR,

• This is not strictly necessary for porting the OpenJDK-specific Reference processing code from the lxr branch to master.
• It will be necessary for LXR or RC-based plans.

Assuming LXR works in the lxr branch, these two points contradict each other. Can you explain this a bit more?

Also, in

• This is not necessary for implementing the SATB barrier

And later you said that we can have RefScanPolicy::SatbBarrier. All these give me a feeling that the proposal is not necessary, as a binding usually can figure out what how to scan ref based on mutator/GC, or the stages in a GC. But it is a 'nice-to-have' thing to make the API more clear. Is this understanding correct?

[wks]

There are a few things that I dont understand about the issue

In TLDR,

• This is not strictly necessary for porting the OpenJDK-specific Reference processing code from the lxr branch to master.
• It will be necessary for LXR or RC-based plans.

Assuming LXR works in the lxr branch, these two points contradict each other. Can you explain this a bit more?

Let met explain. The lxr branch contains two things: (1) the OpenJDK-style reference processor, and (2) the LXR algorithm. If we only want to port the reference processor to master but not the LXR algorithm, there is a trick for distinguishing between StrongClosure and RefUpdate without changing the current mmtk-core API.

The trick is, as you mentioned later, to "figure out what / how to scan refs based on the stage in a GC".

The mmtk-openjdk binding can implement InstanceRefKlass::oop_iterate so that it behaves as RefScanPolicy::StrongClosure by default. This will make it work for the transitive closure of most GCs.

If the VM binding detects that the current plan is PlanSelector::MarkCompact or PlanSelector::Compressor, it can set a global flag in the VMRefClosure bucket so that from that time until the end of GC, InstanceRefKlass::oop_iterate behaves as RefScanPolicy::RefUpdate.

This is why I said it is "not strictly necessary". But it is "nice to have" because this trick will require the VM binding to know more about the phases of concrete GC plans (mark compact and compressor). If we introduce other plans, or if we introduce other use cases of scan_object that are not strictly phase-based, this trick may not work (at least not that well).

Also, in

• This is not necessary for implementing the SATB barrier

And later you said that we can have RefScanPolicy::SatbBarrier. All these give me a feeling that the proposal is not necessary, as a binding usually can figure out what how to scan ref based on mutator/GC, or the stages in a GC. But it is a 'nice-to-have' thing to make the API more clear. Is this understanding correct?

The difference between RefScanPolicy::StrongClosure and RefScanPolicy::SatbBarrier is that the latter should not discover Reference objects. We can still implement SATB barrier with the current API because the VM binding can distinguish the two policies using tls. If it is a mutator, it will behave like RefScanPolicy::SatbBarrier. But as I said above, this is not as nice as explicitly telling the VM binding what the intention is.