test: fix flaky debugger test #2697

Job	Run time
Build slim tarball	1m 48s
aarch64-darwin: with shared libraries	39m 45s
x86_64-linux: with shared libraries	2h 16m 17s
aarch64-linux: with shared libraries	1h 32m 3s
x86_64-darwin: with shared libraries	3h 37m 17s
	8h 7m 10s

Provide feedback