test_runner: fix suite rerun #3345

Job	Run time
Build slim tarball	3m 1s
x86_64-darwin: with shared libraries	52m 29s
aarch64-linux: with shared libraries	28m 27s
aarch64-darwin: with shared libraries	42m 37s
x86_64-linux: with shared libraries	35m 1s
	2h 41m 35s

Provide feedback