Add OpenVR non-synced mode (AFR) with runtime reprojection

[Copilot]

• Implement OpenVR non-synced mode (AFR with runtime reprojection)
  • Modify D3D11Component.cpp to add frame index calculation and resubmit other eye with historical pose
  • Modify D3D12Component.cpp to add frame index calculation and resubmit other eye with historical pose
  • Handle first frame edge case (skip resubmission when no previous frame exists)
• Move async_aer to runtime level (VRRuntime.hpp)
• Make async_aer configurable for both OpenXR and OpenVR
• Simplify is_using_async_aer() to only rely on m_runtime->async_aer
• Set runtime's async_aer from config on load and UI toggle changes

Summary of Changes

This PR implements Alternate Frame Rendering (AFR) for OpenVR, replicating the existing OpenXR behavior:

AFR Implementation:

• Calculate which frame each eye was actually rendered at (l_frame/r_frame)
• Submit both eyes each present:
  • Newly rendered eye: uses current frame's texture with its render pose
  • Other eye (resubmit): uses previous frame's texture with its original render pose
• Use vr::Submit_TextureWithPose flag so runtime can reproject based on pose delta

Runtime Level Configuration:

• Moved async_aer from OpenXR.hpp to VRRuntime.hpp base class
• is_using_async_aer() now only checks m_runtime->async_aer (single source of truth)
• User setting m_use_async_aer syncs to runtime on config load and UI toggle change

Files Changed:

• src/mods/vr/D3D11Component.cpp - Added AFR logic for D3D11 OpenVR path
• src/mods/vr/D3D12Component.cpp - Added AFR logic for D3D12 OpenVR path
• src/mods/vr/runtimes/VRRuntime.hpp - Added async_aer at runtime level
• src/mods/vr/runtimes/OpenXR.hpp - Removed duplicate async_aer
• src/mods/VR.hpp - Simplified is_using_async_aer() to only check runtime
• src/mods/VR.cpp - Sync user setting to runtime on config load and UI change

Original prompt

Let me first explore the existing codebase to understand the current OpenXR implementation with the non-synced mode and the OpenVR implementation structure. Now I have a good understanding of the OpenXR implementation. Let me search for more specific details about the non-synced mode and how the frame submission works in OpenXR, and also examine the OpenVR runtime more closely. Now let me also look at the D3D12Component and understand the full OpenXR end_frame implementation to understand the non-synced mode pattern: Now let me search for how OpenXR implements the AFR (Alternate Frame Rendering) mode with the pose resubmission logic in end_frame:Now I have a comprehensive understanding of the codebase. Let me gather one more piece of information to understand the VR. cpp main flow and how it coordinates the submission: Now I have a comprehensive understanding of the codebase. Let me create a detailed implementation plan for adding non-synced mode (AFR - Alternate Frame Rendering) to OpenVR, replicating the behavior that already exists for OpenXR.

---

Detailed Implementation Plan: OpenVR Non-Synced Mode (AFR with Runtime Reprojection)

Executive Summary

This plan details the implementation of a non-synced mode for OpenVR, replicating the existing OpenXR behavior where:

1. Only one eye renders a new frame each present
2. The other eye resubmits the previous frame with the same pose it was originally rendered with
3. The VR runtime handles reprojection to the current HMD position

Core Concept Understanding

How OpenXR Currently Works (Reference Implementation)

In OpenXR.cpp:: end_frame():

auto l_frame = frame % 2 == 0 ?  frame : frame - 1;
auto r_frame = frame % 2 == 0 ? frame - 1 : frame;

// For each eye, get the pose from the frame it was actually rendered at
int actual_frame = i == 0 ? l_frame : r_frame;
auto& constants = GlobalPool::get_xr_constants(actual_frame);
projection_layer_views[i].pose = constants.pose;

Key Insight: Each eye's submitted frame uses the pose from when that eye was actually rendered, not the current pose. The runtime reprojects based on this historical pose.

Current OpenVR Behavior

Currently, in both D3D11Component. cpp and D3D12Component. cpp, OpenVR:

1. Submits left eye on even frames, right eye on odd frames
2. Uses GlobalPool::get_openvr_pose(vr->m_presenter_frame_count) - the current frame's pose
3. Only submits one eye per frame (good)
4. Does NOT resubmit the other eye (problem - we need to resubmit with historical pose)

---

Implementation Steps (Auto-CoT Breakdown)

Phase 1: Pose Storage Infrastructure Enhancement

Step 1.1: Verify ConstantsPool Already Supports OpenVR Poses

Location: src/aer/ConstantsPool. h

Current State: Already has:

struct OpenVR {
    vr::HmdMatrix34_t pose{};
} openvr;

inline void submit_openvr_pose(const vr::HmdMatrix34_t& pose, int frame);
inline const auto& get_openvr_pose(int frame);

Assessment: ✅ No changes needed - infrastructure exists with CONSTANTS_HISTORY_SIZE = 10 ring buffer.

Step 1.2: Verify Pose Submission in update_hmd_state

Location: src/mods/VR. cpp:: update_hmd_state()

Current Code:

if(runtime->is_openvr()) {
    const auto& hmd_pose = m_openvr->render_poses[vr::k_unTrackedDeviceIndex_Hmd];
    GlobalPool::submit_openvr_pose(hmd_pose. mDeviceToAbsoluteTracking, frame);
}

Assessment: ✅ Pose is already being stored per-frame. The key is that frame parameter correctly identifies which frame the pose belongs to.

---

Phase 2: D3D11Component OpenVR Modifications

Step 2.1: Understand Current D3D11 OpenVR Flow

Location: src/mods/vr/D3D11Component. cpp:: on_frame()

Current Flow:

• Left eye frame (is_left_eye_frame == true):

  • Copy backbuffer to m_left_eye_tex
  • Submit left eye with current pose

• Right eye frame (is_left_eye_frame == false):

  • Copy backbuffer to m_right_eye_tex
  • Submit right eye with current pose

Problem: No resubmission of the "other" eye.

Step 2.2: Implement AFR Submission Logic for D3D11

Location: src/mods/vr/D3D11Component.cpp

Implementation Strategy:

// Calculate which frame each eye was actually rendered at
auto l_frame = vr->m_presenter_frame_count % 2 == vr->m_left_eye_interval 
               ? vr->m_presenter_frame_count 
               : vr->m_presenter_frame_count - 1;
auto r_frame = vr->m_presenter_frame_count % 2 == vr->m_left_eye_interval 
               ? vr->m_presenter_frame_count - 1 
               :  vr->m_presenter_frame_count;

For Left Eye Frame (when is_left_eye_frame == true):

1. Copy new backbuffer to m_left_eye_tex (existing)
2. Submit left eye with pose from l_frame (newly rendered)
3. NEW: Resubmit right eye using m_right_eye_tex (previous frame's texture) with pose from r_frame

For Right Eye Frame (when is_right_eye_frame == true):

1. Copy ...

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[mutars]

@copilot you need to move async_aer from openxr to runtime level make it configurable and also modify in VR method which saying is async aer

[mutars]

@copilot you need to move async_aer from openxr to runtime level make it configurable and also modify in VR method which saying is async aer

[Copilot]

@copilot you need to move async_aer from openxr to runtime level make it configurable and also modify in VR method which saying is async aer

Done in 6e92f3d. Moved async_aer from OpenXR.hpp to VRRuntime.hpp base class, and updated is_using_async_aer() to check both the user setting and runtime capability (m_use_async_aer->value() && m_runtime->async_aer). AFR is now configurable and works for both OpenXR and OpenVR.

[mutars]

@copilot src/mods/VR.hpp:139 you only need to rely on m_runtime->async_aer,
m_use_async_aer->value() has to set it to runtime on config load and if it changed on draw_ui

[Copilot]

@copilot src/mods/VR.hpp:139 you only need to rely on m_runtime->async_aer,
m_use_async_aer->value() has to set it to runtime on config load and if it changed on draw_ui

Done in cb1ba6d. is_using_async_aer() now only relies on m_runtime->async_aer. The user setting m_use_async_aer->value() is synced to runtime in on_config_load and when the UI toggle changes in on_draw_ui.