[Feature Request] Add retro_set_raster_poll API Proposal (To Incubate Beam Raced Outputs)

[mdrejhon]

UPDATE December 2023: Bounty Requirements Reduction

Completing #10758 qualifies for the full bounty published at #6984 ... See latest comment #10758 (comment)

---

Original

We need a very simple per-raster callback function called "retro_set_raster_poll" whose arguments are identical to "retro_set_video_refresh".

Once the centralized hook is done, all emulator modules can just chain it to retro_set_video_refresh by default. This will be default behavior, for beamrace-unaware backwards compatibility

This API is a prerequisite for multiple GitHub items:

• Add Beam Racing/Scanline Sync to RetroArch (aka Lagless VSYNC) #6984 aka Add Beam Racing/Scanline Sync to RetroArch (aka Lagless VSYNC)
• [Feature Request] (BFIv3) Emulate a CRT Electron Beam Via Rolling-Scan BFI #10757 aka (BFIv3) Emulate a CRT Electron Gun Via Rolling-Scan BFI

Either or both may be implemented at some time in the future. Having this feature trailblazed will help.

Stage 1: Start with dummy hooking all emulator modules

Initially, retro_set_raster_poll would just do nothing .... It would never be used. Just add the hook, and make sure that there are do-nothing hooks compatible in all emulator module header files.

Stage 2: Make at least one module support beamraced output

I would pick the NES module since from a quick glance, that module seems to only need potentially maybe just only 10 to 20 lines of modifications to support retro_set_raster_poll (to beam race its output to whomever programs future beamraced outputs within #6984 or #10757).

By having one emulator module already support beamraced output capability, this would make it much easier for develoeprs to begin implementing any kind of beamraced outputs

Behavior for emulator modules that choose to support beamraced outputs,

• Emulator module calls retro_set_raster_poll everytime an emulator scan line is plotted
• It provides the frame buffer containing the incompletely-rendered emulator framebuffer (complete up to the most recently plotted emulator scanline).
• Whomever programs future beamraced outputs can choose to do it in a chunked method (frameslice), per pixel row (scanline method), full-screen (no output beamracing)

You wouldn't have to worry about it, that's the responsibility of whomever programs the handling of retro_set_raster_poll -- the only responsibility for emulator module maintainers is to simply call it when an emulated raster is complete (one pixel row completed into emulator frame buffer).

How many calls per emulator refresh cycle?

• retro_set_raster_poll should be called once every emulator scan line is rendered. The count of calls is equal to the vertical resolution of the original machine (including any overscan area).
• retro_set_video_refresh should be called once per emulator refresh (right after the last call to the retro_set_raster_call for the final scan line).

This allows supporting beamraced outputs in a simple & cross-platform way.

[mdrejhon]

Expected Call-Blocking Behaviours Injected To Emulator Modules

For those people who ask questions about call-blocking behaviours:

Whomever implements this GitHub item don't have to worry about whomever implements beamraced outputs (#6984 or #10757). This information is only provided to provide call-blocking-behaviour understandings, for emulation sync programming.

Handler of retro_set_raster_poll is permitted to block briefly (sub refresh)

Be noted, return from retro_set_raster_poll may be erratic delays (at the microsecond timescales) because the handler (whomever implements #6984 or #10757) may block return for raster interrupt beam racing synchronization purposes.

The module that handles retro_set_raster_poll may be synchronizing to a hardware-based raster, for example, or a refresh cycle interval (e.g. beamracing a 60Hz output to a 360Hz display in segments), as there are multiple hardware-based and software-based beam raced output approaches.

So if it is doing 600-frameslice beam racing, and if it about 1/10th refresh ahead of schedule, it might suddenly decide to block for 1/10th of a refresh cycle, before returning.

This is no different from the existing retro_set_video_refresh which can block for 1 full refresh cycle.

Instead of retro_set_video_refresh blocking for 1 refresh cycle, the combined 200 calls to retro_set_raster_poll for a 200-scanline emulator might potentially have a total sum blockage of one emulator refresh cycle. Basically it's just retro_set_video_refresh blockages fragmented into many tiny pieces.

Expected call blockage of retro_set_raster_poll is balanced

For those wondering how beamraced output handlers will handle it, examples of situations that may be experienced. Whomever later writes the retro_set_raster_poll handler MAY be permitted to do sub-refresh delays, as absolutely necessary for beamraced synchronization

For a 240-pixel-tall emulator frame buffer,

• retro_set_raster_poll creates 240 potential blockages of up to 1/240th of a emulator refresh duration
  _(Example: beam racing to a NVIDIA VRWorks front buffer)
• retro_set_raster_poll creates 10 potential blockages of up to 10/240th of an emulator refresh duration, and 190 immediate zero-delay "returns;"
  (Example use case: VSYNC OFF frameslice beam racing in Add Beam Racing/Scanline Sync to RetroArch (aka Lagless VSYNC) #6984)
• retro_set_raster_poll creates no blockages (240 immediate returns;)
  (Example use case: beam racing not supported by emulator module)
• Other situation, whether consistent blockage per scanline, or batched processing (like frameslice approaches), etc.

The remainder of blocking will occur in retro_set_video_refresh (the normal VSYNC ON blocking behaviour) for whatever retro_set_raster_poll didn't block. It will then just continue to merrily run at correct emulator refresh rates.

For perfect output beamracing, all display output will have been automatically done inside the retro_set_raster_poll with the final afterwards retro_set_video_refresh being NOP (an immediate return;)

Emulator module doesn't have to worry about how output is beamraced

The bottom line is the emulator module shouldn't have to care whether the output display was refreshed via retro_set_raster_poll, or via retro_set_video_refresh, or even both.

It is possible that the beamraced module (which this github item doesn't have to worry about) may automatically switch processing workload balance between retro_set_raster_poll and retro_set_video_refresh ...

For example, a screen being rotated might automatically temporarily disable beamraced sync (if output display scanout direction is not in sync with emulator scanout direction).

Most displays are always sequential scanout, even LCDs, iPads, OLEDs, as seen in high speed videos, www.blurbusters.com/scanout ... which the beamraced handler may or may not piggyback off (#6984 does, but #10757 doesn't necessarily).

[mdrejhon]

Reduced Source Code Bounty Requirements ($500 Bounty)

Did you know Retrotink 4K is a Blur Busters Approved product? I worked with them to add fully adjustable 240Hz BFI to a composite/S-Video/component signal, for output to any 240Hz LCD or OLED. I recommend the new 240Hz OLEDs, since you can reduce 60Hz motion blur by 75% with the 240:60 ratio combined with GtG=nigh near 0. Perfect Sonic Hedgehog with BFI and CRT filters simultaneously...

Retrotink 4K can do everything TestUFO can, including TestUFO Variable Persistence Demo For 240Hz Monitors and even brighten using a HDR nits booster, brighter than LG firmware TV BFI! So I'm way ahead of RetroArch, in an already released BFI product.

So RetroArch, please catch up!

I want to see open source versions.

Also, crosspost:

I recommend #10758 -- "retro_set_raster_poll" API should be added.

By encouraging emulator modules to relay one rendered scanline at a time to RetroArch, this will improve reliability of RetroArch even without frameslice beamracing, because it will keep the GPU's from sleeping, by metering out GPU rendering in tiny time slices (e.g. every 1ms), prevening the GPU from going to sleep between emulator refresh cycles. So this refactoring by pre-requisite #10758 has some MAJOR spinoff benefits, even without frameslice beam racing.

Even emulators that don't beamrace, but execute their frame rendering in realtime (e.g. run in 1ms increments and render directly to GPU framebuffer without letting GPU power-management itself to sleep) also tends to framepace better on 240Hz monitors without needing VRR help, making other things like monolithic BFI more practical.

Obviously, RunAhead is different (not realtime), but it can still benefit; it just runs a big bunch of frames continually (fast raster scanout to many frames), so it will still be a benefit. And you can disable processing in retro_set_raster_poll (dummy hooks) for most modules, and just iterate enhancements later into it. And when 1000Hz OLEDs come, we can forget frameslice beam racing, and simply display 16 incrementally-scanned-out full screen digital refresh cycles per 1 real refresh cycle (much easier to implement low-lag beam racing this way, since full screen refresh cycles are only 1ms). I'm going to see the 480Hz OLED in the invite-only demo room at CES 2024, so ultra high Hz displays are among us, one can't buy a desktop 27" OLED with a refresh rate less than 240Hz; it merely starts there -- even for productivity.

I'd like to see retro_set_raster_poll (#10758) implemented.

Reduced Source Code Bounty Requirements ($500 Bounty)

Special offer: I'll even water-down my existing bounty (#6984) to be paid out only on the mere completion of #10758 if it helps reduce workload. It's such an important germane improvement that enables a lot of spinoff benefits:

Benefits Of Just Merely Programming Only #10758

1. Even without beam racing the destination display, it allows slow continuous metering out to GPU for preventing power management stutters
2. Frameslice beam racing (Add Beam Racing/Scanline Sync to RetroArch (aka Lagless VSYNC) #6984)
3. Ultra high Hz beam racing (480Hz OLEDs allow fully easily displaying 8 partially scanned out emulator frames per 60Hz emulator refresh cycle, and upcoming 1000Hz OLEDs allow fully easily displaying 16 partially scanned out emulator frames per 60Hz emulator refresh cycles)
4. Etc.

cc: @hizzlekizzle

[hizzlekizzle]

oh wow, sounds like those new retrotinks are going to be well worth the money

[mdrejhon]

BountySource bypass

I just found out BountySource is insolvent, so ignore that. I'll put up the money anyway.

$500 Bounty -- directly from Blur Busters / TestUFO

Inquire within, send to squad [at] blurbusters with subject line "Code Bounty: Retroarch", directly referring to this.
[Important: No spam/phisphers. No file attachments from unverified individuals.]

Bounty Conditions

The rule is very simple: Payout will occur via Wise, Interac or PayPal when

1. RetroArch github leads (@hizzlekizzle et al) closes this issue [Feature Request] Add retro_set_raster_poll API Proposal (To Incubate Beam Raced Outputs) #10758
2. At least one emulator module is now calling raster API every raster, even if dummy call
3. I authenticate github UserID who made the pull request for this issue
4. Expiry date set December 31st, 2024
5. Questions are welcome here or at above email