Rolling Scanline Simulation (future improvements)

[LibretroAdmin]

Inviting all stakeholders @MajorPainTheCactus @Ophidon @mdrejhon and others.

This is to discuss further improving the initial groundwork done in this PR - https://github.com/libretro/RetroArch/pull/16282

[Ophidon]

The current problem is that we don't -know- a good way to improve it that doesn't have fairly bad artifacting or other major issues of its own. I personally think the rolling scan feature as it is now, will scare people off BFI thinking it's an entirely useless/broken feature. But I didn't want to stand in the way of merging either, as it isn't my place, and as this code should not inhibit the existing full-frame BFI/shader sub-frame code paths from working as intended.

Some of the best things we know of, for the issues this feature has, are trying to hide the joint lines behind scanlines in CRT filters, and having some overlap between rolling scan sections with brightness adjustment which replaces some of the tearing problem with horizontal strips of less motion blur reduction. Which in and of itself is a pretty apparent visual artifact.

Also, a front-end solution like this wont be aware of what shaders are in use, and the screen resolution and Hz being used will also change where those rolling scan joint lines are in the image. Making trying to build front end code, or a shader specificially meant to be used in conjunction with this feature, need to account for a LOT of different joint line possibilities.

If anyone can provide a solution to where the artifacting is minimal enough to compete with the existing full-frame BFI that has zero inherent artifacting (other than strobing itself being a little annoying, obviously), I am all for it though. There are a few side benefits to the rolling scan method over full-frame BFI when/if it works well. This is where @mdrejhon would be very handy. :)

[hizzlekizzle]

For the record, I find a double ON to be much less obtrusive than a double OFF flicker.

[Ophidon]

Did you mean this response for my last reply on the previous PR regarding the 120hz bfi workaround?

[hizzlekizzle]

Yeah, I just put it here instead of there so we could close the lid on that one and continue discussion of improvements here.

[Ophidon]

A sub-frame shader solution (to that 120hz workaround) wouldn't be able to inject an 'extra' sub-frame like a driver solution could. But I still think it might be better to 'hide' a feature that is purposefully injecting noticeable annoying artifacting in a shader rather than as a front-end option.

So you'd maybe do something more like (100-0)-(100-0)-(50-50)-(0-100)-(0-100) style phase shift on a framecount%(adjustable number of how long you want between phase shifts). And keep in mind framecount intentionally doesn't increment on sub-frames, or sub-frames would mess with anything older that looks at framecount but isn't sub-frame aware.

The 50-50 transition frame might be a less noticeable/annoying transition than just a straight flip like 100-0-0-100? Trading some of the very noticeable change in instantaneous average brightness for some transient motion blur, still annoying but maybe a -little- less distracting.

[MajorPainTheCactus]

Hi Roc-Y I presume this only happens when rolling scan line is turned on?

On Fri, 16 Aug 2024, 17:42 Roc-Y, @.***> wrote:

I don't know why this causes wide black bands in the shader I developed, but I think if the Rolling Scanline Simulation feature only handles the native resolution (e.g. 256*244), then my shader will behave normally. 20240817_003458.jpg (view on web) https://github.com/user-attachments/assets/546e0f9c-5d53-4801-a4f1-ca496e18e89b

— Reply to this email directly, view it on GitHub https://github.com/libretro/RetroArch/issues/16373#issuecomment-2293824917, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVKYGRTCSH6EVMKYMZTDUQDZRYTWLAVCNFSM6AAAAABE53OIC2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTHAZDIOJRG4 . You are receiving this because you were mentioned.Message ID: @.***>

[Roc-Y]

Hi Roc-Y I presume this only happens when rolling scan line is turned on? … On Fri, 16 Aug 2024, 17:42 Roc-Y, @.> wrote: I don't know why this causes wide black bands in the shader I developed, but I think if the Rolling Scanline Simulation feature only handles the native resolution (e.g. 256244), then my shader will behave normally. 20240817_003458.jpg (view on web) https://github.com/user-attachments/assets/546e0f9c-5d53-4801-a4f1-ca496e18e89b — Reply to this email directly, view it on GitHub <#16373 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVKYGRTCSH6EVMKYMZTDUQDZRYTWLAVCNFSM6AAAAABE53OIC2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTHAZDIOJRG4 . You are receiving this because you were mentioned.Message ID: @.*>

There are no black lines after turning off the shader. It seems that as long as the resolution is enlarged in the shader, there will be black lines. It has nothing to do with whether it is a CRT shader.

[mdrejhon]

BTW, in fast horizontal scrolling, there can be tearing artifacts with rolling-scan. You need motion sufficiently fast (about 8 retropixels/frame or faster, which produces 2-pixel-offsets for 4-segment sharp-boundary rolling scan).

This is fixed via using alphablend overlaps. However, gamma-correcting the overlaps can be challenging so that all pixels emit the same number of photons is challenging. And to put fadebehind effects (so that a short-shutter photo of rolling scan looks more similar to a short-shutter photo of a CRT). And even LCD GtG distorts the alphablend overlaps. So alphablend overlaps works best on OLEDs of a known gamma (and doing gamma correction, and disabling ABL). For LCD, sharp-boundary rolling scan is better (and tolerating the tearing artifacts during fast platformers).

Then again using HDR ABL is wonderful, because you can convert SDR into HDR, and use the 25% window size to make rolling-scan strobe much brighter. This improves a lot if you use 8-segment rolling scan (60fps at 480Hz OLED) to reduce HDR window size per refresh cycle, allowing HDR OLED to have a much brighter picture during rolling BFI!

Also, I have a TestUFO version of rolling scan BFI under development that actually simulates the behavior of a CRT beam more accurately (including phosphor fadebehind effect). Related: https://github.com/libretro/RetroArch/issues/10757

[hizzlekizzle]

@songokukakaroto I've been working on subframe BFI shaders that you can try. There's a 240 Hz rolling scan one, but it's not great. As mdrejhon mentioned, the gamma issue means the fades aren't perfect.

[mdrejhon]

I have a surprise coming December 24, 2024 -- the world's most accurate CRT electron beam simulator shader, the "Temporal HLSL" complete enough to close BFIv3 (except for the beamrace part) if integrated into RetroArch.

It looks better than this simulation.

MIT license. @hizzlekizzle, you probably can port it quickly into Retroarch.

You're welcome.

[hizzlekizzle]

@mdrejhon sounds exciting. I can't wait to check it out :D

[mdrejhon]

Even MAME could port it in, if they wish -- @cuavas put a spike on it https://github.com/mamedev/mame/issues/6762 because they said it was a pipe dream. Ah well, RetroArch code contribution it is!

It’s a pie-in-the-sky dream.

It's a reality today.

I just finished working on a new glsl shader for a CRT electron beam simulator I've ever seen, especially when run on a 240Hz OLED.

I'll publicize it with MIT source license via a github you can fork or port on 24th December 2024 -- my release date.

When run on a 240Hz-480Hz OLED to emulate a 60Hz tube, it looks good enough that my shader might become the gensis of CRT-replacement OLEDs for arcade machines of 2030s when it's cheaper to buy these soon-cheaper OLEDs than source nearly-extinct tubes 10 years from now.

[LibretroAdmin]

Sounds great!

[mdrejhon]

Sneak preview!

Slow mo version of realtime shader.

These are genuine screenshots, played back in slow-motion, of it running in real time doing 60fps at 240Hz. Phosphor trailing is adjustable, for brightness-vs-motionclarity. Still works at 2xInHz (Min ~100Hz to emulate PAL CRT, min ~120Hz to emulate NTSC CRT) and up, scales infinitely (>1000Hz+)

Yes, it looks like a slow motion video of a CRT tube. But these are PrintScreen screenshots!

cc: @MajorPainTheCactus @Ophidon @hizzlekizzle

[mdrejhon]

It's out now!

I've published the article: www.blurbusters.com/crt

I've released a MIT-licensed open source shader: www.github.com/blurbusters/crt-beam-simulator

Shadertoy animation demo (for 240Hz) www.shadertoy.com/view/XfKfWd

Can easily adjust settings for 120Hz or 144Hz or 165Hz or 360Hz or 480Hz or 540Hz!

Please implement it into Retroarch. Pretty please.

[mdrejhon]

Discussion also at https://github.com/libretro/RetroArch/issues/10757

[hizzlekizzle]

Porting to RetroArch was a breeze. It's available here now: https://github.com/libretro/slang-shaders/blob/master/subframe-bfi/shaders/crt-beam-simulator.slang and will show up in the online updater in a few minutes/hours.

I replaced the edge-blended version I had made with it, since this one is superior in every way lol.

[mdrejhon]

Porting to RetroArch was a breeze. It's available here now: https://github.com/libretro/slang-shaders/blob/master/subframe-bfi/shaders/crt-beam-simulator.slang and will show up in the online updater in a few minutes/hours.

I replaced the edge-blended version I had made with it, since this one is superior in every way lol.

That was damn fast! Nice Christmas surprise. And I can combine CRT filters simultaneously too? Neat!

You should rename the menu in RetroArch if possible, to at least catch attention -- that it is a new better shader. Also eventually, I'll add a phase-offset since I can reduce the latency of this CRT simulator by probably 1 frameslice (1 native refresh cycle) simply by adding +time as a constant. I need to experiment with my changes in ShaderToy in the coming week (It's Christmas).

But it's absurdly fantastic to see an actual deployment the same day I released my simulation shader!

Which releases will have it? PC, Mac, Linux? Can it also be ported to the mobile app for 120Hz OLED iPhone/iPad too? I notice that the shadertoy works great on those, even if not as good as 240Hz.

TechSpot Readers:

EDIT: TechSpot posted some publicity that contained a permalink to this comment. If you're looking for the original main copy of the shader that will get an improved version in January 2025, please go to my repository: www.github.com/blurbusters/crt-beam-simulator

[hizzlekizzle]

I just asked our Apple guy and he says the subframe stuff is available on nightly builds for iOS but will be included in the upcoming release. It doesn't persist, so you have to re-enable it on each launch, which is a drag, but nothing worth doing is ever easy :)

But yeah, Mac/Win/Lin should be covered.

Thanks for working on this and for designing (and licensing) it around sharing and easy integration into other projects. It was a breeze to port thanks to that foresight and generosity.

[mdrejhon]

Thanks for working on this and for designing (and licensing) it around sharing and easy integration into other projects. It was a breeze to port thanks to that foresight and generosity.

Tim made one of the most important contributions to keep it bright and seam-free (variable-MPRT algorithm).

Niche algorithms tend to be ignored by the display industry, so it's nice we could BYOA (Bring Your Own Algorithm) straight into RetroArch, just supply generic Hz, and the software can do the rest.

And nice that you kept the LCD Saver Mode (maybe add a boolean toggle for it). OLEDs do not require that, and I kind of prefer it be done at the application level to avoid the slewing latency effect [0...1 native Hz]. Not a biggie for 240-480Hz, but turning it off will create constant latency for evenly-divisible refresh rates.

[hizzlekizzle]

Done! https://github.com/libretro/slang-shaders/pull/668

I'm having fun running my subframes up higher than my monitor can push and setting the "FPS Divisor" up accordingly. It looks just like slow-motion camera footage of CRTs. You can get some pretty believable slo-mo captures by pairing it with p_mailn's metaCRT:

[pxdl]

Am I doing something wrong? I can load the other shaders fine, but this one gives me an error:

Edit: Nevermind, had to change the video backend to vulkan and enable shader sub-frames in the settings. Works wonderfully on a 240hz OLED monitor with CRT shaders and HDR turned on.

[hizzlekizzle]

We'd need to see a log of it failing to load to even guess, I'm afraid. This sort of issue is usually handled more effectively via forum/discord/subreddit, though, if you can pop over to one of those.

[Tasosgemah]

How do you load this in RetroArch? When i load the presets nothing happens. I have a 240hz LCD monitor, what other options must i change to make it work?

[pxdl]

@Tasosgemah Enable shader sub-frames in the settings.

[Tasosgemah]

@Tasosgemah Enable shader sub-frames in the settings.

Thanks, it works now. But i assume my monitor isn't good enough for it because even though the motion blur is reduced, it looks really bad. All the colors are very dark, there's some minor ghosting, some noticeable transparent horizontal stripes and random flickering that comes and goes.

[Skyyblaze]

@Tasosgemah Enable shader sub-frames in the settings.

Thanks, it works now. But i assume my monitor isn't good enough for it because even though the motion blur is reduced, it looks really bad. All the colors are very dark, there's some minor ghosting, some noticeable transparent horizontal stripes and random flickering that comes and goes.

@Tasosgemah Something else must be wrong, I have a 160hz monitor set to 120hz for this and it looks super clean and I experience none of this. How many Shader Sub-Frames did you enable?

[Tasosgemah]

It's not my RetroArch settings. I'm getting the same exact issues with the shader toy preview link.

[Tasosgemah]

So i think i fixed the dark visuals issue by using glcore instead of vulkan...

But the horizontal stripes issue is still there:

[mdrejhon]

@Tasosgemah Enable shader sub-frames in the settings.

Thanks, it works now. But i assume my monitor isn't good enough for it because even though the motion blur is reduced, it looks really bad. All the colors are very dark, there's some minor ghosting, some noticeable transparent horizontal stripes and random flickering that comes and goes.

You do need a good screen (preferably OLED or at least a FastIPS LCD) and preferably at least 240Hz.

How To Tweak The Banding Out

If you see appoximately (nativeHz/emulatedHz)-1 bands on the screen (3 band for 240Hz), here's some instructions:

1. Turn of VRR during CRT simulation.
2. Try use SDR instead of HDR temporarily
3. Retroarch settings:

• Adjust Gamma setting
• Adjust Brightness vs Blur setting (brighter)

4. Adjust your monitor to "sRGB" profile, since it is doing realtime gamma2linear() and linear2gamma() every subpixel every native refresh cycle, for the magical Timothy Lottes photon-budget algorithm (spreading overbright pixels over more Hz than dimmer pixels). This linear-knowledge requirement of the CRT simulator is sometimes distorted by custom gamma curves.
5. Try turning off ABL/APL (sometimes it doesn't interfere much, but some models it's bad)
6. FALD MiniLED has problems (FALD backlight lag)

Performance Issues

480Hz Frame Dropping Note

If you only have 1 or 2 bands (especially near the top) at extreme refresh rates like 360Hz or 480Hz, you might have a frame-dropping problem. The CRT simulator is wholly dependant on framerate=Hz, so if any subsection of CRT simulation is missed (2ms time budget per CRT simulated Hz!), you will get glitches.

Also, sometimes horizontal stripes are caused by frameskipping (e.g. dropping 1 out of 4 emulated CRT refresh cycles). Make sure your framerate indicator (use tools like RTSS) equals refresh rate, for proper CRT simulation. I've seen some 480Hz displays show bands because they were framedropping 1 out of 8 simulated CRT refresh cycles.

Check your GPU

If you have erratic flickering at low frame rates, or see multiple bands at what looks like frame-skipped intervals (e.g. 3 bands at 480Hz when there should be 7 or none):

Even a modern Macintosh M1 integrated GPU can do 240Hz CRT beam simulator. You may need to upgrade to a cheap used $200 modern GPU, if you want to keep up at 240Hz+ at 1080p+ with simultaneous CRT filters at the same time.

With basic integrated GPUs (2019-era Intel usually might not keep up) and older GPUs (e.g. old GTX 780's), you might not be able to simulate a CRT. You might get the new 4K60 version of Raspberry Pi GPU to do the CRT simulator up to 1080p240 (estimated), but 1440p240 will be a tall order (etc).

However, the bigger gotcha seems to be how Retroarch is architectured (see below)

Chose lightweight cores

Some heavyweight cycle-accurate cores such as bsnes might hog the CPU/GPU away from the CRT simulator. You may need to switch to a more lightweight emulator (able to finish a frame in less than one native Hz) in order to reliably run the CRT simulator.

Future Ideas

Some experiments is needed to raise the CPU/GPU priority of the CRT simulator to maximum, so that emulation simply run in surge-intervals between the CRT simulated segments, perhaps in a beam-raced way (retro_set_raster_poll in #6984 ...) but not necessarily.

Potential best practices for tweaking Retroarch for more reliable subframe processing:

• GPU shader threads should run at the highest possible priority (even higher priority than emulator core), thread=REALTIME (on both CPU+GPU)
• Core emulation threads should run at BELOW_NORMAL priority or NORMAL priority, any priority level lower than the CPU/GPU shader + Present() threads.
• Emulator core and framebuffer rendering -> BELOW_NORMAL
• CRT simulation framebuffer rendering -> ABOVE_NORMAL or REALTIME
• Present() thread -> busyloop one CPU core + ABOVE_NORMAL or REALTIME
• It's better to stutter the emulation (59 at 60fps) than to stutter the CRT simulator

Just a possible optimization opportunity to make this compatible with 480Hz and 1000Hz OLEDs, since emulation will have to be surge-executed in 0.5ms-1ms steps, to allow CRT beam simulator to run, or some improved CPU+GPU multithreading (shader continually auto-runs, etc).

Some more raster skillz:

BTW, want to see raster beamracingin Javascript? I'm the world's first person to do a real-raster Kefrens Bars. You need a super powerful GPU (GTX 1070 or faster, no internal GPUs) capable of at least 2000fps, since tearlines are rasters. https://www.testufo.com/raster51 -- only works on Microsoft Windows I bet you $1000 for a successful PrintScreen screenshot in Chrome --disable-gpu-vsync mode. You cannot; it's beamraced. Over 100 raster-stabilized tearlines per refresh cycle generating the Kefrens bars pattern, like old-school demos.