Merge error, unable to delete auto-deleted branch

`mp merge` causes the following error

> DELETE https://api.github.com/repos/org/repo/git/refs/heads/my-branch: 422 Reference does not exist []

All of our repos have the `Automatically delete head branches` setting checked for when PRs are merged, which may be the culprit