Run tests in parallel

[abhisrkckl]

Github runners now provide multiple cores.
https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories

Our tests can be sped up significantly if we run them in parallel.

[abhisrkckl]

The test failures seem to be due to astropy cache issues. #1374 may be a solution.

[abhisrkckl]

Here is a summary of what I am doing:

1. Parallelize tests using pytest-xdist. I am using 6 processes here. This is greater than the number of CPU cores available (4), but that's OK because many of our tests are IO-limited rather than CPU-limited. I am getting ~2-3x improvement in test execution time.
2. Re-run failed tests using pytest-rerunfailures. This handles intermittent failures that occur due to random chance.
3. In the macos test, test_toa_selection.py keeps failing in the parallel mode (but is OK when run serially). So I am running that test serially and everything else parallelly for macos. This doesn't affect coverage because we are not computing coverage for macos.
4. I have also updated some tests to use fixtures rather than global variables (these are needed for parallel execution).

[abhisrkckl]

Old Run times (approx)

ubuntu-latest, oldestdeps - 35 min
macos-12, py312-test - 45 min
ubuntu-latest, py312-test-cov - 70 min

New run times (approx)

ubuntu-latest, oldestdeps - 15 min
macos-12, py312-test - 30 min
ubuntu-latest, py312-test-cov - 30 min

These times are for the github runners. They run a lot faster on my laptop.

[dlakaplan]

This is good. I remember we had tried this a while ago but for some reason didn't implement it in CI. But I don't see any reason not to. Even a factor of ~2 change helps.

Is this ready to merge?

[abhisrkckl]

Yes