Skip to content

Optimize CI: conditional jobs, test rebalancing, macOS platform-aware gating#4853

Merged
nanangizz merged 7 commits intomasterfrom
ci/conditional-jobs
Mar 13, 2026
Merged

Optimize CI: conditional jobs, test rebalancing, macOS platform-aware gating#4853
nanangizz merged 7 commits intomasterfrom
ci/conditional-jobs

Conversation

@nanangizz
Copy link
Member

@nanangizz nanangizz commented Mar 12, 2026

Background

PJPROJECT CI currently runs the full test suite on all three platforms (Linux, macOS, Windows) for every PR, regardless of which components changed. This leads to several pain points:

  • Long CI times: A full CI run takes 30-50+ minutes per platform, with some test bundles exceeding 35 minutes each.
  • macOS runner queue bottleneck: macOS runners on GitHub Actions have very limited concurrency. With ~15 jobs per PR, queue wait times regularly exceed 120 minutes before jobs even start, creating a major bottleneck for PR review turnaround.
  • Unnecessary resource usage: Platform-independent changes (e.g., pjsip protocol logic, pjnath) don't need full macOS testing since the code paths are identical to Linux — yet they still consume scarce macOS runner slots.
  • Unbalanced test bundles: Some job bundles were significantly heavier than others (e.g., bundle-1 at 36.5m vs bundle-2 at 18.6m), leading to suboptimal wall-clock times.
  • Flaky CI perception: Long queue times combined with occasional test flakiness make it hard to distinguish real failures from infrastructure issues, slowing down the merge cycle.

This PR addresses these issues by making CI jobs conditional on changed paths, rebalancing test bundles for better parallelism, and applying platform-aware gating to reduce macOS runner pressure.

Feedback, suggestions, and ideas for further CI improvements are very welcome!

Summary

  • Add detect-changes job using dorny/paths-filter@v3 to all 3 CI workflows (Linux, macOS, Windows)
  • Skip irrelevant test jobs on PRs based on which component paths changed
  • Push to master always runs all jobs (github.event_name == 'push' check)
  • macOS optimization: only run full test suite for pjlib/pjmedia changes (platform-specific code); other changes get build-only + CMake verification, with Linux CI providing full test coverage
  • Rebalance test jobs: move pjmedia-test from pjsua bundles to pjsip bundles for better time distribution
  • Fix ioqueue callback lock CI: default is now no-lock, so CI tests the opposite (with-lock)
  • Merge macOS CMake Debug+Release into a single job to save a runner slot

macOS conditional test strategy

macOS-specific code resides in pjlib (OS abstractions, ioqueue/kqueue, sockets, threading, SSL) and pjmedia (audio/video devices, native codecs, AEC). Other components are platform-independent protocol logic covered by Linux CI.

PR changes macOS behavior Linux behavior
pjlib, pjmedia, tests, CI config Full test suite (~15 jobs) Full test suite (per-component)
pjlib-util, pjnath, pjsip, pjsip-apps Build-only + CMake (2 jobs) Full test suite (per-component)
Push to master Always full suite Always full suite

Linux/Windows job conditions by component dependency

Tests run Triggered by changes to
pjlib only core, tests, ci
pjlib-util, pjnath core, pjlib-util, pjnath, tests, ci
pjmedia core, pjlib-util, pjmedia, tests, ci
pjsip core, pjlib-util, pjsip, tests, ci
pjmedia + pjsip core, pjlib-util, pjmedia, pjsip, tests, ci
pjsua (full stack) any source component, tests, ci

Test job rebalancing — Linux OpenH264+VPX

Job Before After
bundle-1 (pjsua) pjmedia-test (15.2m) + pjsua-test (21.3m) = 36.5m pjsua-test (21.3m) = 21.3m
bundle-2 (pjlib,util,pjnath) 18.6m 18.6m (no change)
bundle-3 (pjsip) pjsip-test serial (20.8m) + parallel (15.2m) = 36.0m pjmedia-test (15.2m) + pjsip-test (36.0m) = ~51m

Note: Linux bundle-3 gets heavier but gains the correct pjmedia-pjsip-deps condition, so it skips on pjsua-only or pjnath-only PRs.

Test job rebalancing — Linux Default

Job Before After
bundle-1 (pjsua) pjmedia-test (3.6m) + pjsua-test (21.0m) = 24.6m pjsua-test (21.0m) = 21.0m
bundle-3 (pjsip) pjsip-test (14.4m) = 14.4m pjmedia-test (~3.6m) + pjsip-test (14.4m) = ~18.0m

Test job rebalancing — macOS OpenH264+VPX

Job Before After
bundle-1 (pjsua) pjmedia-test (7.7m) + pjsua-test (24.2m) = 32.0m pjsua-test (24.2m) = 24.2m
bundle-2 (util,pjnath) pjlib-util-test (3.1m) + pjnath-test (17.0m) = 20.1m pjlib-test (9.8m) + pjlib-util-test (3.1m) + pjnath-test (17.0m) = 29.9m
bundle-3 (pjlib,pjsip) pjlib-test (9.8m) + pjsip-test (19.2m) = 29.0m pjmedia-test (7.7m) + pjsip-test (19.2m) = 26.9m

Max job time: 32.0m → 29.9m (better balanced: 24/30/27m)

Windows CI test times (no rebalancing needed)

Job Tests Time
OpenSSL / pjlib,util,pjmedia pjlib (4.6m) + util (3.1m) + pjmedia (0.9m) 8.6m
OpenSSL / pjnath pjnath (12.6m) 12.6m
OpenSSL / pjsip,pjsua pjsip (14.9m) 14.9m
VPX+SChannel / pjlib,util,pjmedia,pjnath pjlib (4.4m) + util (3.1m) + pjmedia (3.1m) + pjnath (0.0m) 10.7m
VPX+SChannel / pjsip pjsip (17.0m) 17.0m
IOCP / pjlib pjlib (4.7m) 4.7m
iocp-cb-with-lock / pjlib,pjsua pjlib (4.4m) + pjsip (9.3m) 13.7m

Windows pjmedia is already separate from pjsua — no changes needed.

Expected savings

  • Docs-only PR: only build-only jobs run (~3 jobs across all OSes), all test jobs skipped
  • pjsip-only or pjnath-only PR: full Linux+Windows tests, macOS build-only (2 jobs vs 15) — saves ~13 Mac runner slots and ~2h queue wait
  • pjlib or pjmedia PR: full test suite on all platforms (as before)

Test plan

  • Verify detect-changes job runs and produces correct outputs on a PR
  • Verify all jobs run on push to master
  • Verify pjlib/pjmedia PR triggers full macOS tests
  • Verify pjsip-only PR skips macOS tests (only build-only + CMake)
  • Verify build-only jobs always run regardless of changes

Co-Authored-By: Claude Code

Use dorny/paths-filter to detect which components changed in PRs
and skip irrelevant test jobs. Build-only jobs always run. Push
to master always runs everything.

Co-Authored-By: Claude Code
Compute pre-combined dependency groups (pjlib-deps, pjnath-deps,
pjmedia-deps, pjsip-deps, pjnath-pjmedia-deps, pjnath-pjsip-deps,
any-src) in detect-changes job so each test job's if-condition is
a single readable line instead of 5-9 OR'd output checks.

Co-Authored-By: Claude Code
Move pjmedia-test from bundle-1 (pjsua) to bundle-3 (pjsip) across
Linux and Mac CI workflows for better time balancing. On Mac, also
move pjlib-test back to bundle-2 to avoid overloading bundle-3.

Add pjmedia-pjsip-deps aggregate output for the new combined bundles.
Configure --enable-libwebrtc-aec3 for Linux default bundle-3 and add
-DHAS_VID_CODEC_TEST=0 for video-enabled bundle-3 jobs.

Co-Authored-By: Claude Code
PJ_IOQUEUE_CALLBACK_NO_LOCK is now 1 by default, so the CI special
config job should test the opposite (with-lock, i.e. value 0).
Rename jobs from cb-no-lock to cb-with-lock accordingly.

Co-Authored-By: Claude Code
Reduces Mac runner slot usage by running both build types
sequentially in one job instead of using a matrix strategy.

Co-Authored-By: Claude Code
macOS-specific code resides in pjlib (OS abstractions, ioqueue/kqueue,
sockets, threading, SSL) and pjmedia (audio/video devices, native
codecs, AEC). Other components (pjlib-util, pjnath, pjsip, pjsip-apps)
are platform-independent and already covered by Linux CI.

Replace per-component conditions with a single mac-full-tests gate
that triggers on pjlib, pjmedia, tests, or CI config changes. For
other changes (e.g. pjsip-only or pjnath-only PRs), only build-only
and CMake jobs run on macOS.

Co-Authored-By: Claude Code
@nanangizz nanangizz changed the title Add conditional CI job execution based on changed paths Optimize CI: conditional jobs, test rebalancing, macOS platform-aware gating Mar 12, 2026
@nanangizz nanangizz added this to the release-2.17 milestone Mar 13, 2026
@nanangizz nanangizz requested a review from Copilot March 13, 2026 03:12
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes PJPROJECT CI by adding path-based change detection and conditional job execution across Linux, macOS, and Windows to reduce unnecessary work (especially on scarce macOS runners) and rebalance test bundles for better overall wall-clock time.

Changes:

  • Add a detect-changes job (using dorny/paths-filter@v3) and gate test jobs based on aggregated “dependency group” outputs.
  • Rebalance several test bundles by moving pjmedia-test from some pjsua bundles into pjsip bundles.
  • Adjust IOQueue callback-lock coverage in CI by flipping the tested default (CI now explicitly tests the “with-lock” configuration).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File Description
.github/workflows/ci-linux.yml Adds change detection + conditional job gating; rebalances Linux test bundles (moves pjmedia-test); updates IOQueue callback-lock job naming/config.
.github/workflows/ci-mac.yml Adds macOS platform-aware gating (full tests only for platform-sensitive changes) and rebalances OpenH264 bundles; merges CMake Debug+Release into one job.
.github/workflows/ci-win.yml Adds change detection + conditional job gating; renames/updates IOCP callback-lock CI job to test with-lock configuration.

You can also share your feedback on Copilot code review. Take the survey.

Avoid potential ambiguity with hyphens in GitHub Actions expression
output names by using underscores (e.g. pjlib_deps, any_src,
mac_full_tests, pjlib_util, pjsip_apps). Actual file paths and
job IDs are unchanged.

Co-Authored-By: Claude Code
@nanangizz nanangizz requested a review from sauwming March 13, 2026 04:11
Copy link
Member

@sauwming sauwming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great idea! The CI tests have been painfully slow recently.

@nanangizz nanangizz merged commit 2b7c8b5 into master Mar 13, 2026
52 checks passed
@nanangizz nanangizz deleted the ci/conditional-jobs branch March 13, 2026 05:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants