Handle rate limit errors from Slack on new alert creation (#1986) #2241

Job	Run time
units / 🧪 Unit Tests	13m 36s
typecheck / typecheck	5m 23s
publish-webapp / publish	3m 24s
publish-worker / build (coordinator)	1m 55s
publish-worker / build (docker-provider)	1m 20s
publish-worker / build (kubernetes-provider)	1m 22s
	27m 0s

Provide feedback