Skip to content

Better CI for UI tests#4002

Merged
wmathurin merged 14 commits intoforcedotcom:devfrom
wmathurin:better_ci
Mar 10, 2026
Merged

Better CI for UI tests#4002
wmathurin merged 14 commits intoforcedotcom:devfrom
wmathurin:better_ci

Conversation

@wmathurin
Copy link
Contributor

@wmathurin wmathurin commented Mar 6, 2026

  • Fixed publication step (test.xcresult is not available for download after tests have run)
  • Timeouts are configurable from the environment (and CI uses longer timeouts)
  • Misc fixes

ios: ${{ matrix.ios }}
xcode: ${{ matrix.xcode }}
secrets: inherit
# ios-nightly:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll revert that - that allows me to test in my fork (see https://github.com/wmathurin/SalesforceMobileSDK-iOS/actions/workflows/nightly.yaml)

@github-actions
Copy link

github-actions bot commented Mar 6, 2026

TestsPassed ☑️SkippedFailed ❌️
SalesforceSDKCore iOS ^18 Test Results594 ran593 ✅1 ❌
TestResult
SalesforceSDKCore iOS ^18 Test Results
DomainDiscoveryCoordinatorTests.testMalformedCallbackURL()❌ failure

@codecov
Copy link

codecov bot commented Mar 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 66.85%. Comparing base (d8a07b9) to head (3b43b4d).
⚠️ Report is 26 commits behind head on dev.

Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #4002      +/-   ##
==========================================
- Coverage   66.89%   66.85%   -0.05%     
==========================================
  Files         254      254              
  Lines       22427    22427              
==========================================
- Hits        15003    14994       -9     
- Misses       7424     7433       +9     
Components Coverage Δ
Analytics 70.78% <ø> (ø)
Common 69.76% <ø> (ø)
Core 59.58% <ø> (-0.07%) ⬇️
SmartStore 74.70% <ø> (ø)
MobileSync 87.41% <ø> (ø)
see 4 files with indirect coverage changes
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link

github-actions bot commented Mar 6, 2026

TestsPassed ☑️SkippedFailed ❌️
SalesforceSDKCore iOS ^26 Test Results594 ran592 ✅2 ❌
TestResult
SalesforceSDKCore iOS ^26 Test Results
DomainDiscoveryCoordinatorTests.testSpecialCharactersInLoginHint()❌ failure
SFUserAccountManagerTests.testUserAccountEncoding❌ failure

@wmathurin
Copy link
Contributor Author

Here is what it looks like: https://github.com/wmathurin/SalesforceMobileSDK-iOS/actions/runs/22809167271

  • Still a lot of flappers (they all run locally).
  • The total counts / failure counts is a bit misleading (tests are run up to 3 times, and when a test fails its 3 runs, that counts as 3 failures).
  • The test.xcresult (with screenshots) is published and can be downloaded for analysis.

@wmathurin wmathurin marked this pull request as ready for review March 9, 2026 20:12
@wmathurin wmathurin merged commit 07036dc into forcedotcom:dev Mar 10, 2026
22 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants