build.d: Add auto-tester-build

[MoonlightSentinel]

Draft PR because this should not be merged before #10611 (or another implementation of parallel building) is merged.
Multiple invocations of build.d are mutually exclusive anyway (using a lockfile)

[MoonlightSentinel]

This failure seems to be caused by small configuration change when creating all targets from build.d. The unittest failure shows sporadically in master when building using a VS Command Prompt.

../dmd/generated/windows/release/32/dmd.exe -conf= -c -o- -Isrc -Iimport -Hfimport\core\sync\barrier.di src\core\sync\barrier.d 

core.exception.AssertError@dmd\root\bitarray.d(170): unittest failure
----------------
0x00742E89 in _d_unittestp
0x006548CC in void dmd.root.bitarray.__modtest()
0x0074AF3A in int core.runtime.runModuleUnitTests().__foreachbody1(object.ModuleInfo*)
1/21 unittests FAILED

[rainers]

core.exception.AssertError@dmd\root\bitarray.d(170): unittest failure

I guess this happens because Bitarray is broken: with size_t = uint, (300+31)/32 = 10*4 bytes are allocated for a length of 300 bits, but (300+31)/8 = 41 bytes are compared.

[MoonlightSentinel]

I guess this happens because Bitarray is broken

Unittests are not run for Win32, temporary fixup before we move this to build.d: #10614

Could not reliably reprodure the unittest failure using the Makefiles so i will fix it in this PR.

[MoonlightSentinel]

core.exception.AssertError@dmd\root\bitarray.d(170): unittest failure

I guess this happens because Bitarray is broken: with size_t = uint, (300+31)/32 = 10*4 bytes are allocated for a length of 300 bits, but (300+31)/8 = 41 bytes are compared.

Thanks. I changed (len + BitsPerChunk - 1) / 8 to ((len + BitsPerChunk - 1) / BitsPerChunk) * ChunkSize) because it yields different results for integers contrary to regular math:

Old:
32-Bit: (300 + 32 - 1) / 8 = 331 / 8 => 41
64-Bit: (300 + 64 - 1) / 8 = 363 / 8 => 45

New;
32-Bit: ((300 + 32 - 1) / 32) * 4 = (331 / 32) * 4 => 10 * 4 = 40
64-Bit: ((300 + 64 - 1) / 64) * 8 = (363 / 64) * 8 => 5 * 8 = 40

@WalterBright Fixed that error in the third commit using dedicated functions to calculate the amount of bytes (and chunks). Would you approve using them throughout bitarray.d to avoid such errors in the future (and make the code more readable)?

[rainers]

I can't say much about build.d, but the changes to bitarray.d are good (you could use chunks() in some places, too). Maybe move these changes to a separate PR?

[MoonlightSentinel]

I can't say much about build.d, but the changes to bitarray.d are good (you could use chunks() in some places, too). Maybe move these changes to a separate PR?

Can do, I wanted to keep them in this PR until they are approved because they wont be tested on windows otherwise.

[MoonlightSentinel]

@rainers See #10620

[dlang-bot]

Thanks for your pull request and interest in making D better, @MoonlightSentinel! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.
Please verify that your PR follows this checklist:

• My PR is fully covered with tests (you can see the coverage diff by visiting the details link of the codecov check)
• My PR is as minimal as possible (smaller, focused PRs are easier to review than big ones)
• I have provided a detailed rationale explaining my changes
• New or modified functions have Ddoc comments (with Params: and Returns:)

Please see CONTRIBUTING.md for more information.

---

If you have addressed all reviews or aren't sure how to proceed, don't hesitate to ping us with a simple comment.

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub fetch digger
dub run digger -- build "master + dmd#10612"

[MoonlightSentinel]

auto-tester-build should be good to go (I can split off enabling unittests for windows in another PR if preferred)

[PetarKirov]

Why is copying the final exe necessary? And also why is it necessary to explicitly pass those flags in the previous step?

[MoonlightSentinel]

Why is copying the final exe necessary?

I don't like this either but the test suite uses dmd/src/dmd.exe

And also why is it necessary to explicitly pass those flags in the previous step?

The executable was previously built by reldmd which invokes make recursively and overrides these flags. Otherwise dmd.exe would be built with -unittest -cov enabled.

[PetarKirov]

Thanks for the explanations!

I don't like this either but the test suite uses dmd/src/dmd.exe

Just like the posix.mak files don't expect dmd to be available at this path for the test suite, we should fix the Win32.mak files as well. This obviously outside of the scope of this PR, so I suggest leaving a FIXME comment explaining the situation.

The executable was previously built by reldmd which invokes make recursively and overrides these flags. Otherwise dmd.exe would be built with -unittest -cov enabled.

I think this should be fixed as well (perhaps in a future PR).
Like other targets that build dmd, whether it's build in release mode or debug + unittest, should be configurable based to env variables like ENABLE_RELEASE. We should run the test suite with both asserts enabled and disabled.

[MoonlightSentinel]

You're welcome. I've added FIXMEs for both issues and removed the ENABLE_RELEASE override (DDEBUG is depreacted anyway).

This implies a slight change as the test suite is now run with bound-checks and asserts instead of optimizations but we can add it as an additional run 'from the outside' .

[PetarKirov]

Yes, what I meant that whether optimizations are enabled or disabled should be disabled from outside.

[MoonlightSentinel]

Ahh, then I'll would suggest keeping the current behaviour and changing it in another PR