-
Notifications
You must be signed in to change notification settings - Fork 78
Open
Description
CI is unreliable: tests fail randomly, then pass without code changes. Multiple issues:
- Tests run 2-3 times per job (VM twice, Chrome twice)
- Separate coverage jobs duplicate all test runs
- macOS times out after 15 minutes
- Mobile emulators are flaky
We plan to ship it in two parts:
Structural Cleanup:
- Add concurrency group (cancel superseded runs)
- Remove macos-14 (duplicate of macos-15)
- Merge linux-coverage and windows-coverage into main jobs
- Fix redundant test runs (VM/Chrome now run once per job)
- Make iOS/Android non-blocking (continue-on-error: true)
Emulator Reliability:
The goal is to make mobile tests deterministic by enabling retries and improving infrastructure.
Android
- Downgrade API 36 to 34 (36 has a known emulator issue; need to evaluate if it was upgraded for a specific reason)
- Cache AVD snapshots (actions/cache@v4)
- Add headless flags: -no-window -gpu swiftshader_indirect -noaudio
- Add 3-attempt retry loop
iOS
- Add 2-attempt retry wrapper (nick-fields/retry@v3)
Future Discussion:
- Debug macOS WASM hang (causes 15min timeout)
- Evaluate if Firefox is needed on all platforms
- Consider Linux-only coverage
Open for opinions
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels