test_runner: fix rerun ambiguous test failures #2364

Job	Run time
Build slim tarball	1m 35s
x86_64-darwin: with shared libraries	46m 20s
aarch64-darwin: with shared libraries	32m 53s
x86_64-linux: with shared libraries	27m 14s
aarch64-linux: with shared libraries	22m 33s
	2h 10m 35s

Provide feedback