-
Notifications
You must be signed in to change notification settings - Fork 7
ci: add Agent Sanity Check workflow and local verification scripts #426
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kirre-bylund
wants to merge
3
commits into
copilot-dev
Choose a base branch
from
feat/ca-compile
base: copilot-dev
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,111 @@ | ||
| # Verifying Changes: Compilation & Tests | ||
|
|
||
| This repo ships as a Unity UPM package. Unity C# must be verified through the Unity | ||
| Editor's compilation pipeline — `dotnet build` does **not** work on this codebase. | ||
|
|
||
| --- | ||
|
|
||
| ## Cloud coding agent (GitHub Copilot agent / CI) | ||
|
|
||
| When you push commits to a work branch the **`Compile Check`** workflow | ||
| (`.github/workflows/compile-check.yml`) runs automatically and verifies: | ||
|
|
||
| 1. All SDK C# code compiles without errors (including Samples). | ||
| 2. The configured Unity **editmode** tests pass (see the workflow for exact coverage). | ||
|
|
||
| ### Workflow to follow after each batch of commits | ||
|
|
||
| 1. Push your work branch (e.g. `feat/my-feature`, `fix/something`). | ||
| 2. Navigate to the **Actions** tab → **Compile Check** → your branch's run. | ||
| 3. Wait for it to complete and confirm it is **green** before opening a PR. | ||
|
|
||
| If the workflow fails: | ||
|
|
||
| 1. **Job Summary** (fastest) — the run's Summary page shows a compact | ||
| `Compilation Errors` table with file path, line number, and error code/message. | ||
| This is the primary place to read errors; no log-digging required. | ||
| 2. **Annotations** — the Summary page also lists inline annotations (e.g. | ||
| `sdk/Runtime/Game/Requests/TriggersRequests.cs#L3`) that link directly to the | ||
| offending line. These also appear in the PR diff view. | ||
| 3. **Raw log** (fallback) — if neither of the above is present the compile step itself | ||
| may have crashed before producing output; click the failed step and search for | ||
| `compilationhadfailure: True` to find the relevant section. | ||
|
|
||
| Fix the reported errors and push again. | ||
|
|
||
| > The full CI suite (`run-tests-and-package.yml`) runs on PRs to `dev` and `main`. | ||
| > The compile check is a smaller, faster subset for in-progress work branches. | ||
|
|
||
| --- | ||
|
|
||
| ## Local (human developer or local Copilot instance) | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| 1. Unity is installed locally (any Unity **2019.2+** version works; best to match the | ||
| project's minimum version in `package.json`). | ||
| 2. You have a `unity-dev-settings.json` at the repo root (gitignored). | ||
| 3. Python 3 is installed and available on your `PATH` as `python3` (used by the | ||
| local verification scripts to parse `unity-dev-settings.json`). | ||
|
|
||
| ### One-time setup | ||
|
|
||
| Open or create `unity-dev-settings.json` and fill in the two fields: | ||
|
|
||
| ```json | ||
| { | ||
| "unity_executable": "<absolute path to Unity binary>", | ||
| "test_project_path": "<relative path to a Unity project including the SDK>" | ||
| } | ||
| ``` | ||
|
|
||
| **`unity_executable` examples by platform:** | ||
|
|
||
| | Platform | Example path | | ||
| |---|---| | ||
| | macOS | `/Applications/Unity/Hub/Editor/2022.3.22f1/Unity.app/Contents/MacOS/Unity` | | ||
| | Windows | `C:\Program Files\Unity\Hub\Editor\2022.3.22f1\Editor\Unity.exe` | | ||
| | Linux | `/opt/unity/Editor/Unity` | | ||
|
|
||
| **`test_project_path`**: leave empty to let the script auto-create a temporary project | ||
| that references the SDK. Set to an absolute path only if you already maintain a | ||
| dedicated local Unity project that points at this SDK via a local package reference. | ||
|
|
||
| ### Running the check | ||
|
|
||
| **Linux / macOS (bash):** | ||
| ```bash | ||
| .github/scripts/verify-compilation.sh | ||
| ``` | ||
|
|
||
| **Windows (PowerShell):** | ||
| ```powershell | ||
| .github\scripts\verify-compilation.ps1 | ||
| ``` | ||
|
|
||
| The script will: | ||
|
|
||
| 1. Read `unity-dev-settings.json`. | ||
| 2. Create a temporary Unity project at `Temp~/VerificationProject` (gitignored) that | ||
| references the SDK as a local package and includes the Samples. | ||
| 3. Launch Unity in batch mode with `-batchmode -nographics -quit`. | ||
| 4. Print Unity's compilation output filtered to errors and warnings. | ||
| 5. Exit `0` on success, `1` on any compilation error. | ||
|
|
||
| --- | ||
|
|
||
| ## What counts as "verified" | ||
|
|
||
| A change is verified when **either** of the following is true: | ||
|
|
||
| - The `Compile Check` CI workflow is green on your branch, **or** | ||
| - The local verification script exits `0`. | ||
|
|
||
| **Additionally** ensure: | ||
|
|
||
| - No new `error CS` compiler errors appear. | ||
| - No existing public API signatures were changed without going through the deprecation | ||
| flow described in `.github/instructions/style-guide.md`. | ||
|
|
||
| Running the full integration tests (`run-tests-and-package.yml`) is a CI-only step and | ||
| is not required for local verification. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,235 @@ | ||
| #Requires -Version 5.0 | ||
| <# | ||
| .SYNOPSIS | ||
| Verifies that the LootLocker Unity SDK compiles without errors. | ||
|
|
||
| .DESCRIPTION | ||
| Reads unity-dev-settings.json from the repo root, optionally creates a temporary | ||
| Unity project referencing the SDK, then runs Unity in batch mode to check | ||
| compilation. | ||
|
|
||
| See .github/instructions/verification.md for setup instructions. | ||
| #> | ||
|
|
||
| $ErrorActionPreference = 'Stop' | ||
|
|
||
| $RepoRoot = (Resolve-Path (Join-Path $PSScriptRoot "..\..")).Path | ||
| $SettingsFile = Join-Path $RepoRoot "unity-dev-settings.json" | ||
| $TempProject = Join-Path $RepoRoot "Temp~\VerificationProject" | ||
| $LogFile = Join-Path $TempProject "compilation.log" | ||
|
|
||
| function Write-Step { param([string]$Msg) Write-Host $Msg } | ||
| function Write-Ok { param([string]$Msg) Write-Host $Msg -ForegroundColor Green } | ||
| function Write-Fail { param([string]$Msg) Write-Host $Msg -ForegroundColor Red } | ||
| function Write-Warn { param([string]$Msg) Write-Host $Msg -ForegroundColor Yellow } | ||
|
|
||
| Write-Step "=========================================" | ||
| Write-Step " LootLocker SDK - Compilation Check" | ||
| Write-Step "=========================================" | ||
| Write-Step "" | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # 1. Load settings | ||
| # --------------------------------------------------------------------------- | ||
| if (-not (Test-Path $SettingsFile)) { | ||
| Write-Warn "SETUP REQUIRED: 'unity-dev-settings.json' not found at repo root." | ||
| Write-Step " Create unity-dev-settings.json with your Unity path." | ||
| Write-Step " See .github/instructions/verification.md for the required format and examples." | ||
| exit 1 | ||
| } | ||
|
|
||
| $Settings = Get-Content $SettingsFile -Raw | ConvertFrom-Json | ||
| $UnityExe = $Settings.unity_executable | ||
| $CustomProject = $Settings.test_project_path | ||
|
|
||
| if ([string]::IsNullOrWhiteSpace($UnityExe)) { | ||
| Write-Fail "ERROR: 'unity_executable' is empty in unity-dev-settings.json." | ||
| exit 1 | ||
| } | ||
| if (-not (Test-Path $UnityExe)) { | ||
| Write-Fail "ERROR: Unity executable not found: $UnityExe" | ||
| exit 1 | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # 2. Helper: create / refresh the temporary verification project | ||
| # --------------------------------------------------------------------------- | ||
| function Initialize-TempProject { | ||
| Write-Step "Creating temporary verification project at Temp~/VerificationProject ..." | ||
|
|
||
| if (Test-Path $TempProject) { Remove-Item $TempProject -Recurse -Force } | ||
| New-Item -ItemType Directory -Path (Join-Path $TempProject "Assets") -Force | Out-Null | ||
| New-Item -ItemType Directory -Path (Join-Path $TempProject "Packages") -Force | Out-Null | ||
| New-Item -ItemType Directory -Path (Join-Path $TempProject "ProjectSettings") -Force | Out-Null | ||
|
|
||
| $SdkRef = $RepoRoot -replace '\\', '/' | ||
| $nl = [char]10 | ||
| $manifestContent = '{' + $nl + ' "dependencies": {' + $nl + ' "com.lootlocker.lootlockersdk": "file:' + $SdkRef + '"' + $nl + ' }' + $nl + '}' | ||
| [IO.File]::WriteAllText((Join-Path $TempProject 'Packages\manifest.json'), $manifestContent) | ||
|
|
||
| $psContent = '%YAML 1.1' + $nl + '%TAG !u! tag:unity3d.com,2011:' + $nl + '--- !u!129 &1' + $nl + 'PlayerSettings:' + $nl + ' companyName: LootLockerSDKVerification' + $nl + ' productName: LootLockerSDKVerification' | ||
| [IO.File]::WriteAllText((Join-Path $TempProject 'ProjectSettings\ProjectSettings.asset'), $psContent) | ||
|
|
||
| $SamplesPath = Join-Path $RepoRoot "Samples~\LootLockerExamples" | ||
| if (Test-Path $SamplesPath) { | ||
| Copy-Item $SamplesPath (Join-Path $TempProject "Assets\") -Recurse -Force | ||
| } | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # 3. Determine project path | ||
| # --------------------------------------------------------------------------- | ||
| if (-not [string]::IsNullOrWhiteSpace($CustomProject) -and (Test-Path $CustomProject)) { | ||
| $ProjectPath = (Resolve-Path $CustomProject).Path | ||
| Write-Step "Using custom project: $ProjectPath" | ||
|
|
||
| # Delete only the LootLocker compiled output artifacts so Tundra is forced to | ||
| # recompile the SDK from source. Deleting the entire Bee folder crashes Unity; | ||
| # deleting only outputs is safe — Tundra detects missing outputs and rebuilds them. | ||
| Write-Step "Removing cached LootLocker assemblies to force recompilation..." | ||
| $artifactsPath = Join-Path $ProjectPath "Library\Bee\artifacts" | ||
| Get-ChildItem $artifactsPath -Recurse -Filter "*lootlocker*" -ErrorAction SilentlyContinue | Remove-Item -Force -ErrorAction SilentlyContinue | ||
| $scriptAssemblies = Join-Path $ProjectPath "Library\ScriptAssemblies" | ||
| Get-ChildItem $scriptAssemblies -Filter "*lootlocker*" -ErrorAction SilentlyContinue | Remove-Item -Force -ErrorAction SilentlyContinue | ||
| } else { | ||
| Initialize-TempProject | ||
| $ProjectPath = $TempProject | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # 4. Run Unity in batch mode | ||
| # --------------------------------------------------------------------------- | ||
| function Invoke-UnityCompile { | ||
| param([string]$ProjectDir) | ||
| Write-Step "" | ||
| Write-Step "Unity: $UnityExe" | ||
| Write-Step "Project: $ProjectDir" | ||
| Write-Step "Log: $LogFile" | ||
| Write-Step "" | ||
|
|
||
| # Ensure log directory exists; remove any stale log from a previous run. | ||
| if (-not (Test-Path (Split-Path $LogFile))) { New-Item -ItemType Directory -Path (Split-Path $LogFile) -Force | Out-Null } | ||
| if (Test-Path $LogFile) { Remove-Item $LogFile -Force } | ||
|
|
||
| $prevEAP = $ErrorActionPreference | ||
| $ErrorActionPreference = 'Continue' | ||
| & $UnityExe -batchmode -nographics -projectPath $ProjectDir -logFile $LogFile -quit | ||
| $script:unityExitCode = if ($LASTEXITCODE -ne $null) { $LASTEXITCODE } else { 1 } | ||
| $ErrorActionPreference = $prevEAP | ||
| } | ||
|
|
||
| $script:unityExitCode = 1 | ||
| Invoke-UnityCompile $ProjectPath | ||
|
|
||
| # Wait for Unity to finish writing the log. Unity child processes (e.g. LicensingClient) | ||
| # may hold the file open after the main process exits. Poll until the log contains | ||
| # Unity's end-of-session marker. A full first-time compile can take 60-90 seconds so we | ||
| # wait up to 3 minutes. We also require the file size to be stable for 3 consecutive | ||
| # reads (1.5 s) before accepting the content, since LicensingClient may still be writing | ||
| # small amounts after the main process has exited. | ||
| $logContent = "" | ||
| $lastSize = -1 | ||
| $stableRuns = 0 | ||
| for ($i = 0; $i -lt 360; $i++) { | ||
| Start-Sleep -Milliseconds 500 | ||
| if (-not (Test-Path $LogFile)) { continue } | ||
| try { | ||
| $stream = [System.IO.File]::Open($LogFile, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read, [System.IO.FileShare]::ReadWrite) | ||
| $reader = [System.IO.StreamReader]::new($stream, $true) # $true = detect encoding from BOM | ||
| $logContent = $reader.ReadToEnd() | ||
| $reader.Close(); $stream.Close() | ||
|
|
||
| $currentSize = $logContent.Length | ||
| if ($currentSize -eq $lastSize) { $stableRuns++ } else { $stableRuns = 0 } | ||
| $lastSize = $currentSize | ||
|
|
||
| # Stop once Unity's final line is present AND the file hasn't grown for 3 checks | ||
| if (($logContent -match "Application will terminate|Exiting batchmode") -and $stableRuns -ge 3) { break } | ||
| } catch { } | ||
| } | ||
|
|
||
| # If the custom project crashed on startup (Package Manager never ran — log is tiny with | ||
| # no compilation output), fall back to the temp project automatically. | ||
| if ($ProjectPath -ne $TempProject -and ($logContent.Length -lt 5000 -or $logContent -notmatch "Package Manager")) { | ||
| Write-Warn "Custom project did not open correctly (startup crash). Falling back to temporary project." | ||
| Write-Warn "To fix: open '$ProjectPath' in the Unity Editor once, then re-run." | ||
| Initialize-TempProject | ||
| $ProjectPath = $TempProject | ||
| Invoke-UnityCompile $ProjectPath | ||
|
|
||
| $lastSize = -1; $stableRuns = 0; $logContent = "" | ||
| for ($i = 0; $i -lt 360; $i++) { | ||
| Start-Sleep -Milliseconds 500 | ||
| if (-not (Test-Path $LogFile)) { continue } | ||
| try { | ||
| $stream = [System.IO.File]::Open($LogFile, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read, [System.IO.FileShare]::ReadWrite) | ||
| $reader = [System.IO.StreamReader]::new($stream, $true) | ||
| $logContent = $reader.ReadToEnd() | ||
| $reader.Close(); $stream.Close() | ||
| $currentSize = $logContent.Length | ||
| if ($currentSize -eq $lastSize) { $stableRuns++ } else { $stableRuns = 0 } | ||
| $lastSize = $currentSize | ||
| if (($logContent -match "Application will terminate|Exiting batchmode") -and $stableRuns -ge 3) { break } | ||
| } catch { } | ||
| } | ||
| } | ||
|
|
||
| $logLines = $logContent -split "`n" | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # 5. Report results | ||
| # --------------------------------------------------------------------------- | ||
| Write-Step "" | ||
| Write-Step "--- Compilation result -------" | ||
|
|
||
| $compileErrors = $logLines | Select-String -Pattern "error CS\d+" | Select-String -NotMatch "Licensing::" | ||
| $tundraSuccess = ($logLines | Select-String -Pattern "Tundra build success").Count -gt 0 | ||
| $tundraFailure = ($logLines | Select-String -Pattern "Tundra build failure|Tundra build failed").Count -gt 0 | ||
|
|
||
| if ($compileErrors) { | ||
| $compileErrors | ForEach-Object { Write-Host $_.Line } | ||
| } | ||
|
|
||
| Write-Step "-----------------------------------" | ||
| Write-Step "" | ||
|
|
||
| # Determine outcome from log content: | ||
| # - Any "error CS####" line => compilation failed | ||
| # - "Tundra build failed" marker => compilation failed | ||
| # - No compiler errors and Unity exit code 0 => compilation succeeded (even if no Tundra marker) | ||
| # - "Tundra build success" found => compilation succeeded (Unity may still exit non-zero | ||
| # due to unrelated project setup issues unrelated to the SDK) | ||
| if ($compileErrors.Count -gt 0) { | ||
| Write-Fail "COMPILATION FAILED ($($compileErrors.Count) compiler error(s))" | ||
| Write-Step "Full log: $LogFile" | ||
| exit 1 | ||
| } | ||
| elseif ($tundraFailure) { | ||
| # Explicit Tundra failure should be treated as a hard failure regardless of exit code | ||
| if ($logLines) { | ||
| $logLines | Select-String -Pattern "error CS\d+|Scripts have compiler errors|error:" | Select-String -NotMatch "Licensing::" | | ||
| ForEach-Object { Write-Host $_.Line } | ||
| } | ||
| Write-Fail "COMPILATION FAILED (Tundra build failed)" | ||
| Write-Step "Full log: $LogFile" | ||
| exit 1 | ||
| } | ||
| elseif ($unityExitCode -eq 0) { | ||
| # Treat a clean Unity exit with no compiler errors as success, even if we did not see a Tundra marker | ||
| Write-Ok "COMPILATION SUCCEEDED" | ||
| } | ||
| elseif ($tundraSuccess) { | ||
| Write-Ok "COMPILATION SUCCEEDED" | ||
| Write-Warn "Note: Unity exited with code $unityExitCode after compilation (likely unrelated project setup - not an SDK issue)." | ||
| } | ||
| else { | ||
| # Non-zero Unity exit code with no clear Tundra success/failure marker; surface diagnostics and fail | ||
| if ($logLines) { | ||
| $logLines | Select-String -Pattern "error CS\d+|Scripts have compiler errors|error:" | Select-String -NotMatch "Licensing::" | | ||
| ForEach-Object { Write-Host $_.Line } | ||
| } | ||
| $reason = "exit code: $unityExitCode" | ||
| Write-Fail "COMPILATION FAILED ($reason)" | ||
| Write-Step "Full log: $LogFile" | ||
| exit 1 | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.