Implement foveated ISR pipeline scaffolding and ExampleUE integration

[Copilot]

Implemented the 4-view foveated ISR pipeline scaffolding and a sample Unreal Engine integration.

• Foveated components: Added StereoEmulator, ViewInjector, FoveatedAtlas, and VisibilityCache with viewport/RT handling and safety checks.
• Example UE integration: Added ExampleUE mod scaffolding (entry, renderer hooks, camera hooks, offsets stub, SDK header) with ImGui UI wiring and static singletons, including Unreal-style frame lifecycle where engine frame counts advance in BeginFrame.
• Offsets/docs: Clarified placeholder signature patterns and retained the design spec for onboarding.

Build note: cmake -S . -B build still fails in this environment due to missing OpenXR xlib backend headers/dependencies.

Original prompt

I'll analyze the mutars/acsvr repository to understand the integration pattern, then provide a condensed, actionable design specification. Now I have a clear understanding of the acsvr integration pattern. Let me create a condensed, agent-friendly specification document with the example integration folder structure.

---

4-View Foveated ISR Pipeline - Design Specification

Requirements

Prerequisites

• UE SDK headers generated (via UE4SS dump or manual RE)
• vrframework as submodule/dependency
• Game-specific memory offsets for hook targets

Folder Structure for Game Integration

src/games/{GameName}/
├── {GameName}Entry.h          // Mod entry point, inherits from Mod
├── {GameName}Entry. cpp
├── {GameName}RendererModule.h // Frame/render tick hooks
├── {GameName}RendererModule.cpp
├── {GameName}CameraModule.h   // View/projection matrix hooks
├── {GameName}CameraModule.cpp
├── memory/
│   └── offsets.h              // Game-specific pattern/offset definitions
└── sdk/
    └── {GameName}SDK.h        // Reversed/generated engine structs

---

Architecture (Flat)

D3D12Hook (vtable/pointer) --> ViewInjector --> FoveatedAtlas (double-height RT)
       |                            |                   |
       v                            v                   v
SetViewports/SetRTs         StereoEmulator(4 views)   VisibilityCache
                                   |
                                   v
                            ISR Draw Pairs:  A(0+1), B(2+3)

Atlas Layout:  [FovealL|FovealR] top, [PeriphL|PeriphR] bottom

---

Core Components

1. StereoEmulator

Manages 4 emulated views with projection/frustum data.

#pragma once
#include <array>
#include <glm/glm. hpp>
#include "mods/vr/runtimes/VRRuntime.hpp"

namespace foveated {

enum class ViewType : uint8_t {
    FOVEAL_LEFT_PRIMARY = 0, FOVEAL_RIGHT_SECONDARY = 1,
    PERIPHERAL_LEFT_PRIMARY = 2, PERIPHERAL_RIGHT_SECONDARY = 3
};

struct EmulatedView {
    glm::mat4 projection, view;
    glm:: vec4 frustumPlanes[6];
    D3D12_VIEWPORT viewport;
    float fovScale;
    ViewType type;
    uint32_t stereoPassMask; // 0x1=Primary, 0x2=Secondary
};

class StereoEmulator {
public:
    static StereoEmulator& get();
    void initialize(runtimes::VRRuntime* runtime);
    void beginFrame(int frameIndex);
    bool isStereoActive() const { return m_stereoActive; }
    const EmulatedView& getView(ViewType t) const { return m_views[(size_t)t]; }
    std::array<D3D12_VIEWPORT, 4> computeAtlasViewports(uint32_t w, uint32_t h) const;
    void configureFOV(float fovealDeg, float peripheralDeg);
private:
    void buildViewMatrices(int frame);
    void computeFrustumPlanes(EmulatedView& v);
    std::array<EmulatedView, 4> m_views;
    runtimes::VRRuntime* m_runtime = nullptr;
    bool m_stereoActive = false;
    float m_fovealFov = 40.f, m_peripheralFov = 110.f;
    float m_fovealScale = 1.f, m_peripheralScale = 0.5f;
};
} // namespace foveated

2. ViewInjector

Hooks D3D12 viewport/RT calls to redirect to atlas.

#pragma once
#include "StereoEmulator. hpp"
#include <d3d12.h>

namespace foveated {

class ViewInjector {
public: 
    static ViewInjector& get();
    void install(class D3D12Hook* hook);
    void setAtlasRT(ID3D12Resource* rt, D3D12_CPU_DESCRIPTOR_HANDLE rtv);
private:
    void onSetViewports(ID3D12GraphicsCommandList5* cmd, UINT num, const D3D12_VIEWPORT* vps);
    void onSetRenderTargets(ID3D12GraphicsCommandList5* cmd, UINT num,
        const D3D12_CPU_DESCRIPTOR_HANDLE* rtvs, BOOL single, D3D12_CPU_DESCRIPTOR_HANDLE* dsv);
    void injectFovealPair(ID3D12GraphicsCommandList5* cmd);
    void injectPeripheralPair(ID3D12GraphicsCommandList5* cmd);
    
    D3D12Hook* m_hook = nullptr;
    ID3D12Resource* m_atlasRT = nullptr;
    D3D12_CPU_DESCRIPTOR_HANDLE m_atlasRTV{};
    std::array<D3D12_VIEWPORT, 4> m_atlasViewports;
    bool m_isRenderingFoveal = false;
};
} // namespace foveated

3. FoveatedAtlas

Double-height render target for 4-view output.

#pragma once
#include <d3d12.h>
#include <wrl/client.h>

namespace foveated {

struct AtlasConfig {
    uint32_t fovealWidth = 1440, fovealHeight = 1600;
    uint32_t peripheralWidth = 720, peripheralHeight = 800;
    DXGI_FORMAT format = DXGI_FORMAT_R10G10B10A2_UNORM;
};

class FoveatedAtlas {
public:
    static FoveatedAtlas& get();
    bool initialize(ID3D12Device* dev, const AtlasConfig& cfg);
    void shutdown();
    ID3D12Resource* getTexture() const { return m_atlas. Get(); }
    D3D12_CPU_DESCRIPTOR_HANDLE getRTV() const { return m_rtv; }
    uint32_t getTotalWidth() const { return m_cfg.fovealWidth * 2; }
    uint32_t getTotalHeight() const { return m_cfg.fovealHeight + m_cfg.peripheralHeight; }
    void transitionToRT(ID3D12GraphicsCommandList* cmd);
    void transitionToSRV(ID3D12GraphicsCommandList...

</details>



<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs.

[mutars]

@copilot it wasn't task to document, it was a task to execute impliementaiton plan, please continue

[Copilot]

@copilot it wasn't task to document, it was a task to execute impliementaiton plan, please continue

Implemented the foveated pipeline components plus ExampleUE integration scaffolding in d136bfa.

[mutars]

@copilot src/games/ExampleUE/ExampleUERendererModule.cpp:33 you need to integrate unreal engine frame count life cycle that was for enknown engine

[Copilot]

@copilot src/games/ExampleUE/ExampleUERendererModule.cpp:33 you need to integrate unreal engine frame count life cycle that was for enknown engine

Adjusted to increment Unreal frame counters in BeginFrame and keep BeginRender for rendering/HMD updates; commit e1426eb.