Skip to content

Improve CI reliability and efficiency #255

@HamdaanAliQuatil

Description

@HamdaanAliQuatil

CI is unreliable: tests fail randomly, then pass without code changes. Multiple issues:

  • Tests run 2-3 times per job (VM twice, Chrome twice)
  • Separate coverage jobs duplicate all test runs
  • macOS times out after 15 minutes
  • Mobile emulators are flaky

We plan to ship it in two parts:

Structural Cleanup:

  • Add concurrency group (cancel superseded runs)
  • Remove macos-14 (duplicate of macos-15)
  • Merge linux-coverage and windows-coverage into main jobs
  • Fix redundant test runs (VM/Chrome now run once per job)
  • Make iOS/Android non-blocking (continue-on-error: true)

Emulator Reliability:
The goal is to make mobile tests deterministic by enabling retries and improving infrastructure.

Android

  • Downgrade API 36 to 34 (36 has a known emulator issue; need to evaluate if it was upgraded for a specific reason)
  • Cache AVD snapshots (actions/cache@v4)
  • Add headless flags: -no-window -gpu swiftshader_indirect -noaudio
  • Add 3-attempt retry loop

iOS

  • Add 2-attempt retry wrapper (nick-fields/retry@v3)

Future Discussion:

  • Debug macOS WASM hang (causes 15min timeout)
  • Evaluate if Firefox is needed on all platforms
  • Consider Linux-only coverage

Open for opinions

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions