test_runner: fix suite rerun #3303

Job	Run time
Build slim tarball	2m 24s
x86_64-linux: with shared libraries	37m 16s
aarch64-linux: with shared libraries	39m 25s
x86_64-darwin: with shared libraries	2h 45m 30s
aarch64-darwin: with shared libraries	1h 20m 36s
	5h 25m 11s

Provide feedback