Don't parallelize scanning for small codebases

[seddonym]

The recently-introduced multiprocessing work had slowed down the test suite, because we were spawning multiple processes for each functional test that scanned relatively small packages.

This introduces a constant defining the minimum number of modules before using multiprocessing, returning the test suite to its previous speed.

pytest --benchmark-skip
Before: 13.85s.
After: 4.91s

We can see there's a regression here for scanning Django with 15 cache misses: as it happens, the same size as the testpackage that is sped up. I think the reason for that is that the test package has very little AST parsing to do as the Python modules are almost empty, whereas the Django benchmark includes 15 quite complex modules. We could adjust this a bit (or even run it at a higher threshold when running tests), but given that scanning 50 modules in a single process is still quite fast, I doubt it will impact users in practice. We can easily revisit this later if need be.

[codspeed-hq]

CodSpeed Instrumentation Performance Report

Merging #207 will degrade performances by 54.35%

_{Comparing small-codebase-build-optimization (059a5d9) with master (30e36bc)}

Summary

❌ 1 regressions
✅ 21 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	BASE	HEAD	Change
❌	test_build_django_from_cache_a_few_misses[15]	174.2 ms	381.6 ms	-54.35%