Implement 4-View Foveated ISR Pipeline with Game Integration Template

[Copilot]

Implementation Plan for 4-View Foveated ISR Pipeline

Phase 1: Core Foveated Rendering Components ✅

• Create src/mods/foveated/ directory structure
• Implement StereoEmulator.hpp - Manages 4 emulated views with projection/frustum data
• Implement StereoEmulator.cpp - Implementation of StereoEmulator
• Implement ViewInjector.hpp - Hooks D3D12 viewport/RT calls to redirect to atlas
• Implement ViewInjector.cpp - Implementation of ViewInjector
• Implement FoveatedAtlas.hpp - Double-height render target for 4-view output
• Implement FoveatedAtlas.cpp - Implementation of FoveatedAtlas
• Implement VisibilityCache.hpp - Shares culling results between foveal/peripheral pairs
• Implement VisibilityCache.cpp - Implementation of VisibilityCache

Phase 2: Example Game Integration Template ✅

• Create src/games/ExampleUE/ directory structure
• Implement ExampleUEEntry.h - Mod entry point
• Implement ExampleUEEntry.cpp - Entry point implementation
• Implement ExampleUERendererModule.h - Frame/render tick hooks
• Implement ExampleUERendererModule.cpp - Renderer module implementation with proper UE frame sync
• Implement ExampleUECameraModule.h - View/projection matrix hooks
• Implement ExampleUECameraModule.cpp - Camera module implementation
• Create src/games/ExampleUE/memory/offsets.h - Pattern-based offset resolution including GFrameNumber
• Create src/games/ExampleUE/sdk/ExampleUESDK.h - Reversed engine structures including frame counter

Phase 3: Build System Integration ✅

• Files automatically included via GLOB_RECURSE in CMakeLists.txt
• Addressed code review feedback

Phase 4: Validation ✅

• Request code review - addressed all issues
• Run security checks with codeql_checker - no issues found
• Add comprehensive documentation (FOVEATED_RENDERING.md)
• Address frame lifecycle feedback - implemented proper UE frame synchronization

Summary

Successfully implemented complete 4-View Foveated ISR Pipeline with:

• 1,085 lines of code across 16 files (updated)
• Core foveated rendering components (767 lines)
• Example game integration template (318 lines)
• Engine-specific frame handling: Reads GFrameNumber directly from Unreal Engine for perfect frame synchronization
• Comprehensive documentation
• All code review feedback addressed
• No security vulnerabilities detected

Original prompt

I'll analyze the mutars/acsvr repository to understand the integration pattern, then provide a condensed, actionable design specification. Now I have a clear understanding of the acsvr integration pattern. Let me create a condensed, agent-friendly specification document with the example integration folder structure.

---

4-View Foveated ISR Pipeline - Design Specification

Requirements

Prerequisites

• UE SDK headers generated (via UE4SS dump or manual RE)
• vrframework as submodule/dependency
• Game-specific memory offsets for hook targets

Folder Structure for Game Integration

src/games/{GameName}/
├── {GameName}Entry.h          // Mod entry point, inherits from Mod
├── {GameName}Entry. cpp
├── {GameName}RendererModule.h // Frame/render tick hooks
├── {GameName}RendererModule.cpp
├── {GameName}CameraModule.h   // View/projection matrix hooks
├── {GameName}CameraModule.cpp
├── memory/
│   └── offsets.h              // Game-specific pattern/offset definitions
└── sdk/
    └── {GameName}SDK.h        // Reversed/generated engine structs

---

Architecture (Flat)

D3D12Hook (vtable/pointer) --> ViewInjector --> FoveatedAtlas (double-height RT)
       |                            |                   |
       v                            v                   v
SetViewports/SetRTs         StereoEmulator(4 views)   VisibilityCache
                                   |
                                   v
                            ISR Draw Pairs:  A(0+1), B(2+3)

Atlas Layout:  [FovealL|FovealR] top, [PeriphL|PeriphR] bottom

---

Core Components

1. StereoEmulator

Manages 4 emulated views with projection/frustum data.

#pragma once
#include <array>
#include <glm/glm. hpp>
#include "mods/vr/runtimes/VRRuntime.hpp"

namespace foveated {

enum class ViewType : uint8_t {
    FOVEAL_LEFT_PRIMARY = 0, FOVEAL_RIGHT_SECONDARY = 1,
    PERIPHERAL_LEFT_PRIMARY = 2, PERIPHERAL_RIGHT_SECONDARY = 3
};

struct EmulatedView {
    glm::mat4 projection, view;
    glm:: vec4 frustumPlanes[6];
    D3D12_VIEWPORT viewport;
    float fovScale;
    ViewType type;
    uint32_t stereoPassMask; // 0x1=Primary, 0x2=Secondary
};

class StereoEmulator {
public:
    static StereoEmulator& get();
    void initialize(runtimes::VRRuntime* runtime);
    void beginFrame(int frameIndex);
    bool isStereoActive() const { return m_stereoActive; }
    const EmulatedView& getView(ViewType t) const { return m_views[(size_t)t]; }
    std::array<D3D12_VIEWPORT, 4> computeAtlasViewports(uint32_t w, uint32_t h) const;
    void configureFOV(float fovealDeg, float peripheralDeg);
private:
    void buildViewMatrices(int frame);
    void computeFrustumPlanes(EmulatedView& v);
    std::array<EmulatedView, 4> m_views;
    runtimes::VRRuntime* m_runtime = nullptr;
    bool m_stereoActive = false;
    float m_fovealFov = 40.f, m_peripheralFov = 110.f;
    float m_fovealScale = 1.f, m_peripheralScale = 0.5f;
};
} // namespace foveated

2. ViewInjector

Hooks D3D12 viewport/RT calls to redirect to atlas.

#pragma once
#include "StereoEmulator. hpp"
#include <d3d12.h>

namespace foveated {

class ViewInjector {
public: 
    static ViewInjector& get();
    void install(class D3D12Hook* hook);
    void setAtlasRT(ID3D12Resource* rt, D3D12_CPU_DESCRIPTOR_HANDLE rtv);
private:
    void onSetViewports(ID3D12GraphicsCommandList5* cmd, UINT num, const D3D12_VIEWPORT* vps);
    void onSetRenderTargets(ID3D12GraphicsCommandList5* cmd, UINT num,
        const D3D12_CPU_DESCRIPTOR_HANDLE* rtvs, BOOL single, D3D12_CPU_DESCRIPTOR_HANDLE* dsv);
    void injectFovealPair(ID3D12GraphicsCommandList5* cmd);
    void injectPeripheralPair(ID3D12GraphicsCommandList5* cmd);
    
    D3D12Hook* m_hook = nullptr;
    ID3D12Resource* m_atlasRT = nullptr;
    D3D12_CPU_DESCRIPTOR_HANDLE m_atlasRTV{};
    std::array<D3D12_VIEWPORT, 4> m_atlasViewports;
    bool m_isRenderingFoveal = false;
};
} // namespace foveated

3. FoveatedAtlas

Double-height render target for 4-view output.

#pragma once
#include <d3d12.h>
#include <wrl/client.h>

namespace foveated {

struct AtlasConfig {
    uint32_t fovealWidth = 1440, fovealHeight = 1600;
    uint32_t peripheralWidth = 720, peripheralHeight = 800;
    DXGI_FORMAT format = DXGI_FORMAT_R10G10B10A2_UNORM;
};

class FoveatedAtlas {
public:
    static FoveatedAtlas& get();
    bool initialize(ID3D12Device* dev, const AtlasConfig& cfg);
    void shutdown();
    ID3D12Resource* getTexture() const { return m_atlas. Get(); }
    D3D12_CPU_DESCRIPTOR_HANDLE getRTV() const { return m_rtv; }
    uint32_t getTotalWidth() const { return m_cfg.fovealWidth * 2; }
    uint32_t getTotalHeight() const { return m_cfg.fovealHeight + m_cfg.peripheralHeight; }
    void transitionToRT(ID3D12GraphicsCommandList* cmd);
    void transitionToSRV(ID3D12GraphicsCommandList...

</details>



<!-- START COPILOT CODING AGENT TIPS -->
---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).

[mutars]

@copilot you need to dig more into frame lifecyle, and ideally you want to gram frame from underal engine directly, as this was done mainly for unknown engines, when you do that you won't need to worry if engine is one frame nehind or not

[Copilot]

Updated the implementation to read the frame counter directly from Unreal Engine's GFrameNumber global variable instead of manually incrementing. This ensures proper frame lifecycle synchronization and eliminates the risk of being one frame behind.

The changes include:

• Added getGFrameNumberAddr() pattern scanning function in memory/offsets.h
• Added GFrameNumber pointer declaration in SDK structures
• Modified ExampleUERendererModule::onBeginRender() to read from GFrameNumber directly
• Falls back to manual increment if GFrameNumber isn't found (for unknown engines)
• Updated documentation to explain the importance of engine-specific frame handling

Commit: 0176e9f