Automatically restart broken unison processes #164
Description
Goal
#147 notes that Unison crashes without warning. EugenMayer/docker-sync#497 notes that there are cases where filesystem/sync stress leads to runaway CPU usage which will lead to a crash. If we detect that runaway state and restart the unison process, the bootstrapping process will take a clean look at the files on both sides of the sync and general grind through the sync instead of getting trapped in thrashing.
How can we handle the restart?
Notes
A few bits of research around this:
- https://github.com/EugenMayer/docker-image-unison/pull/11/files demonstrates using Monit to monitor for performance and use supervisord to restart the unison server process.
- https://asciinema.org/a/148457 is a demonstration of the problem occurring
- For the process inside the container, should we restart unison in the running container or rely on Docker healthchecks and auto-healing such as discussed on stackoverflow to handle that?