`HudElementRegistry#replaceElement` does not work properly to Closed Captions in MC 1.21.9/10

[i5wear]

Minecraft Version: 1.21.9 & 1.21.10

Fabric Loader Version: 0.17.3

Fabric API Version: 0.134.0+1.21.9 & 0.134.1+1.21.10

Description of issue:

I wrapped the VanillaHudElements to adjust their scale and opacity. It works for almost all of them.

When it comes to SUBTITLES (aka Closed Caption), however, the wrapper does not work at all if the client is running.

ONLY when I press ESC to pause, it works, even in a multiplayer server.

I think it should be easy to reproduce by using my mod HUD Manager.

[i5wear]

To make it clear.

[i5wear]

The incoming NeoForge solution for this issue.

https://github.com/neoforged/NeoForge/pull/2741

[Liam-Broome]

@i5wear - Is this still an issue you're able to reproduce?

[i5wear]

@Liam-Broome - Let me have a try:

HudElementRegistry.replaceElement(
    VanillaHudElements.SUBTITLES, original -> (graphics, tracker) -> {
        graphics.pose().pushMatrix();
        graphics.pose().scale(0.5f);
        original.render(graphics, tracker);
        graphics.pose().popMatrix();
    }
);

[i5wear]

Even with the latest version of Fabric Loader (0.18.0) and Fabric API (0.138.3+1.21.10), this issue still exists.

I've figured out its cause:

So, this has to do with the deferred nature of subtitles. If the player in in the game ui (meaning no screen or where Screen#isInGameUi is true, the subtitles are deferred, rendering at different point in time (such as before GUIs or on top of the blurred background in screens). As such, wrapping the element in those cases will do nothing in case of the deferred nature.

This should be fixable if the layer itself as added as the deferred subtitle, but the method of how to do so will probably need a bit of baking.

Quoted from (neoforged/NeoForge#2739).

[i5wear]

This is exactly the mixin that associates to the issue:

@WrapOperation(
    method = {"render(Lnet/minecraft/client/gui/GuiGraphics;Lnet/minecraft/client/DeltaTracker;)V"},
    at = {@At(
    value = "INVOKE",
    target = "Lnet/minecraft/client/gui/Gui;renderSubtitleOverlay(Lnet/minecraft/client/gui/GuiGraphics;Z)V"
    )}
)
private void wrapSubtitlesHud(Gui instance, GuiGraphics context, boolean bl, Operation<Void> renderVanilla, @Local(argsOnly = true) DeltaTracker tickCounter) {
    HudElementRegistryImpl.getRoot(VanillaHudElements.SUBTITLES).render(context, tickCounter, (ctx, tc) -> renderVanilla.call(new Object[]{instance, ctx, bl}));
}

Its operation wrapped is exactly to call the following method:

private void renderSubtitleOverlay(final GuiGraphics graphics, final boolean deferRendering) {
	if (deferRendering) {
		this.deferredSubtitles = () -> this.subtitleOverlay.render(graphics);
	} else {
		this.deferredSubtitles = null;
		this.subtitleOverlay.render(graphics);
	}
}

It is a big problem when the internal SubtitleOverlay#render method will actually call, as all the operations we do on the GuiGraphics (aka DrawContext if using Yarn mapping) is immediate.

[Liam-Broome]

@i5wear - Thank you for the information that is very helpful. I will take a look at it tonight.

[Liam-Broome]

I have tried to reproduce this using the mod that you referenced, but it seems to work for me. If I put scale up on the subtitles the scale will apply during gameplay and stay the same when paused.

[i5wear]

I have tried to reproduce this using the mod that you referenced, but it seems to work for me. If I put scale up on the subtitles the scale will apply during gameplay and stay the same when paused.

Well, I have applied a fix to this issue simply by not to use the method for SUBTITLES, but to mixin SubtitleOverlay.class instead.

I think this is reproduceable by using an earlier version of my mod, like 2.0.0, or trying the following test codes:

HudElementRegistry.replaceElement(
    VanillaHudElements.SUBTITLES, original -> (graphics, tracker) -> {
        graphics.pose().pushMatrix();
        graphics.pose().scale(0.25f);
        original.render(graphics, tracker);
        graphics.pose().popMatrix();
    }
);

Notably, wrapping SubtitleOverlay#render method instead of Gui#renderSubtitleOverlay could be a solution to this issue. However, it causes divergence, as the way to wrap SUBTITLES is different from all other HUD elements.

[i5wear]

@Liam-Broome - Thank you for your patience to look into such a tiny yet complicated issue.

[Liam-Broome]

Created a PR for it.

[Liam-Broome]

@i5wear - Thank you for the help. Sorry it took so long for me to replicate, I focused too much on using your mod than just recreating it using the code you provided 😆

[i5wear]

BTW, this issue also exists in NeoForge and is still not fixed :fire: