ci: partition heavy EGS test modules for parallel execution#77
ci: partition heavy EGS test modules for parallel execution#77
Conversation
Use snforge --partition to split token (4 partitions), minigame, metagame, and registry (2 partitions each) across multiple cheaper runners. This eliminates ubuntu-latest-32 and ubuntu-latest-8 runners for EGS while doubling fuzzer runs from 32 to 64. Jobs: 16 → 22 (all on ubuntu-latest-4/8 instead of 32-core runners) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughReplaces static per-module CI matrices with a dynamic module-catalog-based matrix generator that emits per-entry JSON (including partition and fuzzer_runs). CI workflows now build the matrix in a setup job, pass it to test jobs, support partitioned test execution, and update codecov notification thresholds and docs to match the new matrix entry count. Changes
Sequence Diagram(s)mermaid Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly refactors the Continuous Integration (CI) pipeline for Embeddable Game Standard (EGS) modules. By introducing parallel execution for computationally intensive modules like Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Ignored Files
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Updates CI to run heavy Embeddable Game Standard (EGS) test modules in parallel partitions using snforge test --partition, aiming to improve throughput and reduce reliance on very large runners while keeping Codecov aggregation consistent with the expanded matrix.
Changes:
- Partition EGS-heavy modules in both
main-ciandpr-citest matrices and update job naming to show partition indices. - Adjust Codecov aggregation expectations by updating
after_n_buildsand aligning validation messaging from “modules” → “matrix entries”. - Refresh CI matrix documentation in
AGENTS.mdto reflect partitioning and runner sizing guidance.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
codecov.yml |
Updates after_n_builds to match the new total matrix entry count (including partitions). |
AGENTS.md |
Documents the new partitioned CI matrix and runner guidance for heavy modules. |
.github/workflows/pr-ci.yml |
Generates a partition-aware matrix, updates job naming, and passes --partition to snforge test. |
.github/workflows/main-ci.yml |
Expands the main test matrix with partition fields and applies --partition to test execution. |
.github/workflows/main-ci.yml
Outdated
| - package: game_components_embeddable_game_standard | ||
| module: token | ||
| runner: ubuntu-latest-32 | ||
| fuzzer_runs: 32 | ||
| runner: ubuntu-latest-8 | ||
| fuzzer_runs: 64 | ||
| partition: 1 | ||
| total_partitions: 4 |
There was a problem hiding this comment.
The PR description says it “eliminates all ubuntu-latest-32 and ubuntu-latest-8 EGS runners”, but the EGS token partitions here still run on ubuntu-latest-8. Update the PR description (or adjust the runner choice) so the documented CI changes match what’s actually implemented.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request effectively partitions heavy EGS test modules to improve CI parallelism and efficiency. The configuration changes in codecov.yml and the documentation updates in AGENTS.md are consistent with this goal. However, I've noted a significant reduction in fuzzer runs for the metagame and registry modules as a side effect of this partitioning, which could impact test thoroughness. My review includes a comment with a suggestion to address this.
AGENTS.md
Outdated
| | `embeddable_game_standard` | `metagame` | `ubuntu-latest-4` | 64 | 2 | | ||
| | `embeddable_game_standard` | `registry` | `ubuntu-latest-4` | 64 | 2 | |
There was a problem hiding this comment.
The number of fuzzer runs for the metagame and registry modules has been reduced from 256 to 64 per partition. With 2 partitions each, the total number of fuzzer runs is now 128 (64 * 2), which is a 50% reduction from the previous configuration. This could decrease the effectiveness of fuzz testing for these modules. To restore the original level of fuzzing, consider increasing the runs per partition to 128.
| | `embeddable_game_standard` | `metagame` | `ubuntu-latest-4` | 64 | 2 | | |
| | `embeddable_game_standard` | `registry` | `ubuntu-latest-4` | 64 | 2 | | |
| | `embeddable_game_standard` | `metagame` | `ubuntu-latest-4` | 128 | 2 | | |
| | `embeddable_game_standard` | `registry` | `ubuntu-latest-4` | 128 | 2 | |
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Partition all 11 modules that took >60s into 2 partitions each (only distribution stays at 1 — it ran in 54s) - Remove `scarb build --workspace` from setup job; snforge test already compiles and each test job checks for warnings - Jobs: 22 → 33 (all on ubuntu-latest-4, except token on -8) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/main-ci.yml:
- Around line 161-182: Add an actionlint configuration file that defines the
custom self-hosted runner labels to silence unknown-label warnings: create an
actionlint.yaml file containing a self-hosted-runner block with labels listing
"ubuntu-latest-4" and "ubuntu-latest-8" so actionlint recognizes these runner
labels used in the workflow; ensure the file is committed to the repository root
where actionlint will pick it up.
In `@codecov.yml`:
- Around line 4-7: The codecov.yml entry after_n_builds: 33 is static and causes
PR CI validation failures when pr-ci.yml generates a smaller dynamic matrix
(matrix_count); update the setup by either (A) splitting configuration into two
codecov configs (one for main branch with after_n_builds: 33 and one for PRs
with the correct smaller or absent after_n_builds) or (B) replace the static
after_n_builds usage with Codecov conditional flags/projects so codecov_count is
computed per-run to match pr-ci.yml's dynamic matrix_count; modify codecov.yml
(after_n_builds) and the PR workflow validation logic so they align with
pr-ci.yml's NEED_* dynamic matrix behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: b80d5058-71eb-4ab5-8dce-2d18ca6d5201
📒 Files selected for processing (5)
.github/workflows/main-ci.yml.github/workflows/pr-ci.ymlAGENTS.mdcodecov.ymlpackages/testing/src/constants.cairo
.github/workflows/main-ci.yml
Outdated
| runner: ubuntu-latest-8 | ||
| fuzzer_runs: 64 | ||
| partition: 1 | ||
| total_partitions: 4 | ||
| - package: game_components_embeddable_game_standard | ||
| module: minigame | ||
| module: token | ||
| runner: ubuntu-latest-8 | ||
| fuzzer_runs: 32 | ||
| fuzzer_runs: 64 | ||
| partition: 2 | ||
| total_partitions: 4 | ||
| - package: game_components_embeddable_game_standard | ||
| module: metagame | ||
| module: token | ||
| runner: ubuntu-latest-8 | ||
| fuzzer_runs: 32 | ||
| fuzzer_runs: 64 | ||
| partition: 3 | ||
| total_partitions: 4 | ||
| - package: game_components_embeddable_game_standard | ||
| module: registry | ||
| module: token | ||
| runner: ubuntu-latest-8 | ||
| fuzzer_runs: 32 | ||
| # Metagame | ||
| fuzzer_runs: 64 | ||
| partition: 4 | ||
| total_partitions: 4 |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check if actionlint config exists
if [ -f .github/actionlint.yaml ]; then
echo "actionlint.yaml exists:"
cat .github/actionlint.yaml
else
echo "No actionlint.yaml found. Runner labels may need to be configured."
fi
# Check for any runner configuration
fd -t f 'actionlint' .github/Repository: Provable-Games/game-components
Length of output: 140
🏁 Script executed:
# Search for all runner label references in CI workflows
rg "runner:" .github/workflows/ -A 1 -B 1 | head -100
# Check what runner labels are used throughout the CI
rg "ubuntu-latest" .github/workflows/Repository: Provable-Games/game-components
Length of output: 9603
🌐 Web query:
GitHub Actions standard runner labels ubuntu-latest cores 2024 2025
💡 Result:
Standard GitHub-hosted runner labels (Ubuntu)
ubuntu-latest(x64),ubuntu-24.04(x64),ubuntu-22.04(x64)ubuntu-24.04-arm,ubuntu-22.04-arm(arm64)ubuntu-slim(x64, single-CPU container runner) (docs.github.com)
How many CPU cores does ubuntu-latest have?
GitHub’s “standard runners” have different specs for public vs private repos:
- Public repositories:
ubuntu-latest= 4 CPU cores, 16 GB RAM (docs.github.com) - Private repositories:
ubuntu-latest= 2 CPU cores, 8 GB RAM (docs.github.com)
What did ubuntu-latest point to in 2024 vs 2025?
- GitHub announced
ubuntu-latestwould migrate from Ubuntu 22.04 → 24.04 during Sep 23, 2024 to Oct 30, 2024. (github.blog) - The
actions/runner-imagesannouncement issue states the rollout schedule as Dec 5, 2024 → Jan 17, 2025, making Ubuntu 24.04 the default behindubuntu-latestby completion. (github.com)
Related (not “standard”): larger runner core-count labels
If you meant the explicit core-size labels (larger runners), GitHub provides labels like:
ubuntu-latest-4-cores(and other sizes) (github.blog)
Citations:
- 1: https://docs.github.com/actions/reference/runners/github-hosted-runners
- 2: https://docs.github.com/actions/reference/runners/github-hosted-runners
- 3: https://docs.github.com/actions/reference/runners/github-hosted-runners
- 4: https://github.blog/changelog/2024-09-25-actions-new-images-and-ubuntu-latest-changes
- 5: Ubuntu-latest workflows will use Ubuntu-24.04 image actions/runner-images#10636
- 6: https://github.blog/changelog/2022-10-20-github-actions-larger-hosted-runners-are-now-automatically-created-for-customers/?utm_source=openai
Add actionlint.yaml configuration to suppress runner label warnings.
The repository uses custom runner labels (ubuntu-latest-8 and ubuntu-latest-4) that are not part of GitHub's standard runner set (which uses formats like ubuntu-latest, ubuntu-24.04, or ubuntu-latest-X-cores). Since no actionlint.yaml currently exists, linting tools will flag these as unknown labels. Add the following configuration to suppress these warnings:
# .github/actionlint.yaml
self-hosted-runner:
labels:
- ubuntu-latest-4
- ubuntu-latest-8🧰 Tools
🪛 actionlint (1.7.11)
[error] 161-161: label "ubuntu-latest-8" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2025-vs2026", "windows-2022", "windows-11-arm", "ubuntu-slim", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26-large", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xlarge", "macos-14-large", "macos-14", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
[error] 167-167: label "ubuntu-latest-8" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2025-vs2026", "windows-2022", "windows-11-arm", "ubuntu-slim", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26-large", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xlarge", "macos-14-large", "macos-14", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
[error] 173-173: label "ubuntu-latest-8" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2025-vs2026", "windows-2022", "windows-11-arm", "ubuntu-slim", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26-large", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xlarge", "macos-14-large", "macos-14", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
[error] 179-179: label "ubuntu-latest-8" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2025-vs2026", "windows-2022", "windows-11-arm", "ubuntu-slim", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26-large", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xlarge", "macos-14-large", "macos-14", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/main-ci.yml around lines 161 - 182, Add an actionlint
configuration file that defines the custom self-hosted runner labels to silence
unknown-label warnings: create an actionlint.yaml file containing a
self-hosted-runner block with labels listing "ubuntu-latest-4" and
"ubuntu-latest-8" so actionlint recognizes these runner labels used in the
workflow; ensure the file is committed to the repository root where actionlint
will pick it up.
Replace all non-standard runners (4-core, 8-core) with ubuntu-latest (standard 2-core) by scaling partition counts proportionally: - 4-core modules → 4× partitions (e.g., 2→8) - 8-core EGS modules → 8× partitions (e.g., 4→32 for token) Extract shared module catalog into module-catalog.sh sourced by both main-ci.yml and pr-ci.yml. Update snforge to v0.57.0 for --partition support. Total: 148 jobs across 16 modules, all on standard runners. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GitHub Actions runs bash with set -eo pipefail. The pattern `[ cond ] && fn` exits 1 when the condition is false, killing the step. Use if/then/fi instead. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The snforge CLI was updated to 0.57.0 in .tool-versions but the snforge_std library dependency in Scarb.toml was still pinned to v0.55.0, causing a version mismatch warning. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Scale partition counts to match original runner sizes: - 32-core/8-core modules (EGS) → 8 partitions - 4-core modules (all others) → 4 partitions Total: 80 jobs across 16 modules, all on standard runners. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (1)
codecov.yml (1)
4-7:⚠️ Potential issue | 🟠 Major
after_n_builds: 148is still incompatible with PR CI’s dynamic upload count.
pr-ci.ymlonly uploads coverage for the package groups selected byNEED_*, so a utilities-only or presets-only PR will produce far fewer uploads than 148. With one shared Codecov config, PR notifications/status can end up waiting on builds that were never scheduled.How does Codecov's `notify.after_n_builds` behave when a workflow uploads fewer reports than the configured count, and is a separate PR-vs-main configuration recommended for dynamic GitHub Actions matrices?🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@codecov.yml` around lines 4 - 7, The Codecov notify threshold after_n_builds is set to a fixed 148 which is incompatible with pr-ci.yml's dynamic upload count controlled by NEED_* (some PRs upload far fewer reports), so change the configuration to avoid waiting for nonexistent uploads: either remove or set after_n_builds to 0 (disable waiting) and rely on default behavior, or split configs so main CI uses after_n_builds=148 while PR runs use a PR-specific config with notify disabled; update codecov.yml (after_n_builds) and the PR workflow invocation (pr-ci.yml / NEED_* usage) accordingly so PR runs no longer block waiting for 148 uploads.
🧹 Nitpick comments (1)
.github/workflows/module-catalog.sh (1)
15-20: Droprunnerfrom the catalog or wire it back intoruns-on.Both workflows now hardcode
runs-on: ubuntu-latest, so therunnerproperty emitted here is dead config. That makes future per-module runner changes easy to “update” in the catalog without any effect.♻️ One way to keep the catalog honest
- '. + [range($total) | {package:$p, module:$m, runner:"ubuntu-latest", fuzzer_runs:$f, partition:(. + 1), total_partitions:$total}]') + '. + [range($total) | {package:$p, module:$m, fuzzer_runs:$f, partition:(. + 1), total_partitions:$total}]')🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/module-catalog.sh around lines 15 - 20, The add() function is emitting a dead runner field into INCLUDES (the jq expression in add() adds runner:"ubuntu-latest"), which is unused because workflows hardcode runs-on; remove the runner property from the jq object so the catalog no longer contains stale runner data (or alternatively, if you prefer to preserve per-module runners, wire the emitted runner value into the workflows' runs-on field instead); update the jq expression inside add() (the '. + [range($total) | {package:$p, module:$m, fuzzer_runs:$f, partition:(. + 1), total_partitions:$total}]' part) and ensure any consumers that expected runner are adjusted accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/main-ci.yml:
- Around line 225-237: The test step currently pipes "snforge test ... | tee
/tmp/test.log" so the workflow can return tee's exit code; fix this by enabling
pipefail before the pipeline (add "set -o pipefail" or equivalent at the top of
the same shell block that runs the PARTITION_FLAG/TEST_FILTER variable setup and
the "snforge test ... | tee /tmp/test.log" command) so that the step fails when
"snforge test" fails.
In @.github/workflows/module-catalog.sh:
- Around line 22-65: The matrix is over-sharding modules causing 148 jobs;
update the partition counts in add_egs, add_metagame, add_economy,
add_utilities, and add_presets so they produce the intended ~22 jobs (e.g.,
remove aggressive sharding by setting the partition/count fields to 1 for most
modules and only keep larger splits where explicitly intended), adjusting the
numeric pairs passed to add (the total_partitions/partition_count arguments) to
match the documented rollout and then verify add_all_modules expands to ~22
entries.
In @.github/workflows/pr-ci.yml:
- Around line 411-423: The CI step "Run tests with coverage" currently pipes the
snforge test invocation to tee which masks the test exit status; enable shell
pipefail for that run block so the pipeline returns snforge's exit code (e.g.,
add set -o pipefail or set -euo pipefail at the start of the run script) or
alternatively capture the exit of the snforge test command and explicitly exit
with it after tee (use PIPESTATUS[0] to propagate snforge's exit status) so the
step fails when snforge test fails; apply this change around the snforge test
invocation that writes to /tmp/test.log.
---
Duplicate comments:
In `@codecov.yml`:
- Around line 4-7: The Codecov notify threshold after_n_builds is set to a fixed
148 which is incompatible with pr-ci.yml's dynamic upload count controlled by
NEED_* (some PRs upload far fewer reports), so change the configuration to avoid
waiting for nonexistent uploads: either remove or set after_n_builds to 0
(disable waiting) and rely on default behavior, or split configs so main CI uses
after_n_builds=148 while PR runs use a PR-specific config with notify disabled;
update codecov.yml (after_n_builds) and the PR workflow invocation (pr-ci.yml /
NEED_* usage) accordingly so PR runs no longer block waiting for 148 uploads.
---
Nitpick comments:
In @.github/workflows/module-catalog.sh:
- Around line 15-20: The add() function is emitting a dead runner field into
INCLUDES (the jq expression in add() adds runner:"ubuntu-latest"), which is
unused because workflows hardcode runs-on; remove the runner property from the
jq object so the catalog no longer contains stale runner data (or alternatively,
if you prefer to preserve per-module runners, wire the emitted runner value into
the workflows' runs-on field instead); update the jq expression inside add()
(the '. + [range($total) | {package:$p, module:$m, fuzzer_runs:$f, partition:(.
+ 1), total_partitions:$total}]' part) and ensure any consumers that expected
runner are adjusted accordingly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 7e246f62-697d-43cc-8f53-4789d6c6b9b3
📒 Files selected for processing (7)
.github/workflows/main-ci.yml.github/workflows/module-catalog.sh.github/workflows/pr-ci.yml.tool-versionsAGENTS.mdScarb.tomlcodecov.yml
| - name: Run tests with coverage | ||
| run: | | ||
| if [ "${{ matrix.package }}" = "game_components_${{ matrix.module }}" ]; then | ||
| snforge test -p ${{ matrix.package }} --fuzzer-runs ${{ matrix.fuzzer_runs }} --coverage 2>&1 | tee /tmp/test.log | ||
| else | ||
| snforge test -p ${{ matrix.package }} "::${{ matrix.module }}::" --fuzzer-runs ${{ matrix.fuzzer_runs }} --coverage 2>&1 | tee /tmp/test.log | ||
| PARTITION_FLAG="" | ||
| if [ "${{ matrix.total_partitions }}" != "1" ]; then | ||
| PARTITION_FLAG="--partition ${{ matrix.partition }}/${{ matrix.total_partitions }}" | ||
| fi | ||
|
|
||
| TEST_FILTER="" | ||
| if [ "${{ matrix.package }}" != "game_components_${{ matrix.module }}" ]; then | ||
| TEST_FILTER="::${{ matrix.module }}::" | ||
| fi | ||
|
|
||
| snforge test -p ${{ matrix.package }} $TEST_FILTER --fuzzer-runs ${{ matrix.fuzzer_runs }} $PARTITION_FLAG --coverage 2>&1 | tee /tmp/test.log |
There was a problem hiding this comment.
This test step can also pass on failing snforge runs.
Line 237 pipes the test command into tee, so without set -o pipefail the workflow reports tee’s exit code instead of snforge test’s.
🛠️ Minimal fix
- name: Run tests with coverage
run: |
+ set -o pipefail
PARTITION_FLAG=""
if [ "${{ matrix.total_partitions }}" != "1" ]; then
PARTITION_FLAG="--partition ${{ matrix.partition }}/${{ matrix.total_partitions }}"
fi🧰 Tools
🪛 actionlint (1.7.11)
[error] 226-226: shellcheck reported issue in this script: SC2086:info:11:92: Double quote to prevent globbing and word splitting
(shellcheck)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/main-ci.yml around lines 225 - 237, The test step
currently pipes "snforge test ... | tee /tmp/test.log" so the workflow can
return tee's exit code; fix this by enabling pipefail before the pipeline (add
"set -o pipefail" or equivalent at the top of the same shell block that runs the
PARTITION_FLAG/TEST_FILTER variable setup and the "snforge test ... | tee
/tmp/test.log" command) so that the step fails when "snforge test" fails.
| - name: Run tests with coverage | ||
| run: | | ||
| if [ "${{ matrix.package }}" = "game_components_${{ matrix.module }}" ]; then | ||
| snforge test -p ${{ matrix.package }} --fuzzer-runs ${{ matrix.fuzzer_runs }} --coverage 2>&1 | tee /tmp/test.log | ||
| else | ||
| snforge test -p ${{ matrix.package }} "::${{ matrix.module }}::" --fuzzer-runs ${{ matrix.fuzzer_runs }} --coverage 2>&1 | tee /tmp/test.log | ||
| PARTITION_FLAG="" | ||
| if [ "${{ matrix.total_partitions }}" != "1" ]; then | ||
| PARTITION_FLAG="--partition ${{ matrix.partition }}/${{ matrix.total_partitions }}" | ||
| fi | ||
|
|
||
| TEST_FILTER="" | ||
| if [ "${{ matrix.package }}" != "game_components_${{ matrix.module }}" ]; then | ||
| TEST_FILTER="::${{ matrix.module }}::" | ||
| fi | ||
|
|
||
| snforge test -p ${{ matrix.package }} $TEST_FILTER --fuzzer-runs ${{ matrix.fuzzer_runs }} $PARTITION_FLAG --coverage 2>&1 | tee /tmp/test.log |
There was a problem hiding this comment.
Test failures can be hidden by the tee pipeline.
Line 423 returns tee’s status unless pipefail is enabled, so a failing snforge test partition can still leave the step green.
🛠️ Minimal fix
- name: Run tests with coverage
run: |
+ set -o pipefail
PARTITION_FLAG=""
if [ "${{ matrix.total_partitions }}" != "1" ]; then
PARTITION_FLAG="--partition ${{ matrix.partition }}/${{ matrix.total_partitions }}"
fi🧰 Tools
🪛 actionlint (1.7.11)
[error] 412-412: shellcheck reported issue in this script: SC2086:info:11:92: Double quote to prevent globbing and word splitting
(shellcheck)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/pr-ci.yml around lines 411 - 423, The CI step "Run tests
with coverage" currently pipes the snforge test invocation to tee which masks
the test exit status; enable shell pipefail for that run block so the pipeline
returns snforge's exit code (e.g., add set -o pipefail or set -euo pipefail at
the start of the run script) or alternatively capture the exit of the snforge
test command and explicitly exit with it after tee (use PIPESTATUS[0] to
propagate snforge's exit status) so the step fails when snforge test fails;
apply this change around the snforge test invocation that writes to
/tmp/test.log.
Summary
snforge --partition INDEX/TOTALto split heavy Embeddable Game Standard modules across multiple parallel runnersubuntu-latest-32andubuntu-latest-8EGS runnerscodecov.ymlafter_n_buildsfrom 16 → 22AGENTS.mdCI matrix documentationTest plan
validate-configstep passes (22module:lines ==after_n_builds: 22)test (token 1/4)for partitioned,test (leaderboard)for non-partitioned🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Chores
Documentation